16 ms·

It's interesting to see how much "standardized cross-company interview for software eng" has been consistently a cursed problem in the industry. Unlike airline

It's interesting to see how much "standardized cross-company interview for software eng" has been consistently a cursed problem in the industry. Unlike airline pilots (or I'm certain many other professions), every company in the valley insists on re-interviewing a candidate in their own custom and unique way, instead of trusting some sort of an industry-wide certificate only achievable through a standardized test. Wonder if this will ever be solved. I remember way back in the day where it felt like Triplebyte might finally figure that one out, but it unfortunately never happened.

gjsman-1000 2y ago
Part of me wonders if the recruiting itself is over-engineered anyway. I mean, imagine if you just asked: Implement Bubble Sort, in 2 Languages of your choice, with multithreading or other language-provided parallelism, with the correct number of parallel threads to be most algorithmically efficient Would that really not weed out a lot of people? I think it would. I know the above algorithm is hardly production-ready, but the requirements are easy to understand. (It's also a bit of a trick question - there is no optimal algorithmically efficient number of threads in a bubble sort, only the number of CPU cores in the system.)
- anonymoushn 2y ago
  What's your approach for parallelizing bubble sort in a way that profits from cores beyond the first?
  - thdc 2y ago
    It's probably alternating the comparisons. Compare [0, 1] [2, 3] [4, 5] ... in parallel and swap if necessary, compare/swap [1, 2] [3, 4] [5, 6] ... in parallel, then go back and forth until no more swaps are made - second element in pair is always greater/less than the first. That does suggest that the theoretical ideal number of threads is n / 2 ignoring cores, though you'll also want to consider things like cache line size, cache coherency, and actual problem size so it's probably less. At the end of the day, the important thing would be to actually benchmark your attempts and seeing how it scales with processors/problem size and checking the isoefficiency. I think it was a bad question.
- aidos 2y ago
  What are you trying to achieve here? Are you looking to remove a bunch of competent developers by asking weird trick questions?
  - gjsman-1000 2y ago
    1. It's the dumbest algorithm for sorting possible (bubble sort). Compare two objects, swap them if one is bigger than the other, go down the list, repeat. If a developer doesn't know that algorithm, what algorithm could they possibly know? It's the lowest bar. 2. 2 Languages of your choice is enough to show you aren't a frameworker and can think in more than one box. Doesn't matter if you do it in JavaScript and C#, Go and Rust, Python and Haskell. It's a chance to show off, while being quite obtainable for a competent developer. 3. The parallelism trick question merely shows that you actually understand what you are talking about. A leetcoder might make the trap of assuming it's log(n), or the square root of n elements, or some other overly-thought-through math that is bogus. If you think it through though, bubble sort is simple enough, that it's very easy to realize (in my opinion) why more threads than CPU cores doesn't really help.
    - probably_wrong 2y ago
      > or some other overly-thought-through math that is bogus. So the mathematically-correct answer would be a minus while "in my opinion" would be a plus? I think your interview question brings you to your intended answer: the reason why every company makes their own process is because what's good for you would probably get you an instant fail in NASA and would get you labeled as "experience not relevant" in some research settings.
      - gjsman-1000 2y ago
        > So the mathematically-correct answer would be a minus while "in my opinion" would be a plus? My opinion was only that it would be relatively easy to figure out. As far as I know, it is a certainty that sectioning a bubble sort, more or less than CPU cores, is less efficient. 16 threads on an 8 core CPU just means balancing 2 threads on every CPU core, each doing their own sorting and needing eventual merging back together. There's no way that can be more efficient.
- rachofsunshine 2y ago
  Our coding problem is easier than that (by a fair margin, it's all completely synchronous single-threaded procedural code unless you're doing something extremely weird) and it weeds out the vast majority of applicants. The same was true of all three of the standard coding problems Triplebyte used. They're not quite literal fizzbuzz, but they require - at best - some basic critical thinking and basic language features, and that is a filter that eliminates 90+% of applicants to dev jobs. Now, granted, this is under time pressure. I imagine, given several hours, most could finish it (although maybe even that is overestimating things). But still. There's an old Randall Munroe article quoting a physicist: > The physicist who mentioned this problem to me told me his rule of thumb for estimating supernova-related numbers: However big you think supernovae are, they're bigger than that. and I feel like this applies to recruiting: however bad you think the average applicant is, they're worse than that.
  - josephg 2y ago
    Triplebyte used to send candidates a link to a page which basically listed everything that was in the interview. Most candidates didn’t read it. They could probably have done away with the interview entirely and just put a link at the bottom of the prep material saying “click here to pass the interview”. If anything I think it would have lowered the triplebyte passing rate.
    - judahmeek 2y ago
      Why not try this as an experimental filter then and see what sort of population pass?
      - josephg 2y ago
        Well, triplebyte is dead. That’s one reason not to do the experiment. Also arguably the “customer” for triplebyte was the company that eventually hires the candidate. They’re paying in part for the screening process triplebyte did. We wouldn’t have been doing our job if we skipped the interview part of the process.
  - 2y ago
- zeroCalories 2y ago
  I find picking good questions is hard, and many fall into similar patterns, making them something candidates can practice for. Even your question isn't something I'd necessarily ask on the spot. Many engineers don't use parallelism in their day-to-day work(webdevs). The part about making it efficient is interesting, but feels borderline like a trick question that a good engineer could fumble.
  - gjsman-1000 2y ago
    > Even your question isn't something I'd necessarily ask on the spot. Many engineers don't use parallelism in their day-to-day work(webdevs). The part about making it efficient is interesting, but feels borderline like a trick question that a good engineer could fumble. True, it's more of a backend role question. The reason I threw it in there, is from my assumption a leetcode grinder would be very likely to immediately go, "well, the most efficient number of threads is log (n)" or "the most efficient number of threads is the square root of n" or some other plausible-sounding BS answer. But the reason I chose bubble sort, is that it's so simple to understand, that you can fairly easily (I would hope) figure out there's no benefits to more threads than CPU cores at all, as long as you stop and actually think about what it is doing.
- gamepsys 2y ago
  I don't think we are over-engineering it. You want to "weed out" everyone but the best candidate for the role, or the best candidates for your open roles. It's a very hard problem to identify the "best" person from a group of people. It would be different if all programmers that are good enough are the same, but we all know the skill ceiling on programming is very high. Selecting the best devs is a critical function for any software project.
- LargeWu 2y ago
  It's also a mostly useless problem for determining engineer quality, in many cases. It tests for pure coding ability, when most organizations should be optimizing for engineers that are going to be productive-if-not-spectacular, that can design and build maintainable systems. Could I have written the above problem back in my engineering days? Probably not, since I went years not working with threads. But I also wasn't working on problems that would ever have benefited from knowing that. Most software engineering roles are essentially building CRUD or ETL systems, maybe with a user interface. Any coding problems given should be optimized for weeding out bozos (which are still plentiful), not for weeding out the most people.
- Uehreka 2y ago
  I do a tree-search-ish thing when interviewing people. I’ll start with a super basic question about something beginner-ish on their resume. If they can’t answer that, the interview is politely wrapped up. I’ve eliminated a surprising number of people who had jQuery on their resume by asking them to write code that will make a div with the ID “mydiv” disappear if the user clicks a button with the id “mybutton”. After that I ask a super difficult or niche trivia question like “in CSS, what two overflow properties cannot be used together?” If I’m hiring a mid-level frontend developer and they nail that one, I go “fantastic, great answer, do you have any questions for us?” And the interview can end early. But if they miss that, no sweat, I’ll start asking more mid-level technical questions to figure out where they’re at.
- timfsu 2y ago
  This sounds like what a lot of companies do - except the scaled problem with this approach (and the certification approach of the grandparent) is that most companies want to avoid candidates who've memorized a specific solution, as then they don't get any data about whether they can code anything aside from what was memorized. The other problem is that implementing bubble sort will tell you about their skills in a particular dimension, but being a software engineer these days may look very different depending on the job.
dataminded 2y ago
It's because companies don't want capable, experienced or well-equipped. They want genius and it is really hard to test for genius. Granted, almost nobody that gets through any process is an actual genius....
- onlyrealcuzzo 2y ago
  It's because you have to go through a lengthy process to become a pilot - and tech companies want to be able to hire anyone regardless of training. Combine that with the fact that the upper bound on pay for SWEs is considerably higher than pilots...
- debacle 2y ago
  I have never needed to hire a genius in the last 12 years. In fact, often times I've had to pass on candidates that were overqualified.
- duped 2y ago
  I'd say it's the exact opposite. There are hordes of unqualified people applying to every software dev role imaginable regardless of what you put in the job description or requirements. The tests are there because people are good at lying but bad at faking skills.
  - rachofsunshine 2y ago
    Seriously, if you've never hired before, you have no idea how bad this can get. Here's [1] our practice coding problem. It's quite similar to the one we use on our interview, and not too far from the one Triplebyte used in the past (ours is tuned to be slightly harder at the beginning and slightly easier at the end). The vast majority of candidates, even with some reasonable pre-filtering, do not get past the first step. A very non-trivial number would not even get that far. [1] https://www.otherbranch.com/practice-coding-problem https://www.otherbranch.com/practice-coding-problem
    - itronitron 2y ago
      That is the sort of coding problem I would love to work on as an exercise however the 25 minute time limit would put me off and I wouldn't even start. I enjoy programming and I don't like to feel rushed, and if you're placing that sort of time constraint then you probably aren't a great company to work for.
listenallyall 2y ago
A standardized test exists for lawyers (two, if you count the LSATs) but law firms still interview potential hires.
- saulpw 2y ago
  Do these firms give them their equivalent of an LSAT or the bar exam? AFAIK it's a "non-technical" interview.
- janalsncm 2y ago
  Is law a good example? My understanding is if you didn’t go to a top 14 school (whoever came up with that arbitrary number) it basically forecloses on the best opportunities.
  - evantbyrne 2y ago
    A similar pattern exists in tech startup hiring practices and which ones attract VC funding. Not unusual to see funded startups with founders who have no work experience but Stanford degrees. Before my time at The Atlantic, I had a couple recruiters for no-name startups tell me I didn't have a prestigious enough background to be hired. There is a highly-visible class hierarchy in tech that many people in the industry seem unaware of. Perhaps this is because base salaries are high-enough that the middle class is just happy to be included at all.
    - happyopossum 2y ago
      Don’t listen to recruiters? They lie. I have been at several startups, including a massive unicorn, and now work at a faang. I have no college experience whatsoever.
    - rachofsunshine 2y ago
      100%. I'm not sure if this was always the case or if it was a slow result of tech becoming overrun by finance, but it's a very motivating thing for me. I started a company for a lot of reasons, and this isn't the top one, but I sure would love to show success without playing the class-signalling games that the valley seems overrun with. On the other hand, I'm posting my content post here in part because I know the HN candidate pool is about a trillion times better than I'll get anywhere else. So perhaps I've already lost that battle.
sevensor 2y ago
This process is also pretty much guaranteed never to yield mid-career geniuses at the height of their powers. Those candidates don't go looking for work at all. Work comes looking for them. Why would they go on _any_ jobs platform, ever? Effective filtering of the candidates who actually engage with the platform can, at best, accurately identify the next tier down: effective engineers in mid- and late-career, and inexperienced whiz kids. Not that this is a bad thing; that first category makes the world go 'round.
- josephg 2y ago
  > Those candidates don't go looking for work at all. Work comes looking for them. Why would they go on _any_ jobs platform, ever? Because I don't know what’s out there, or who will give me the best offer. If you’re skilled and in the middle of your career, it’s easy to find a job, but if your options are wide open, a matchmaking service like this with a wide pool of companies is very valuable.
nordsieck 2y ago
> It's interesting to see how much "standardized cross-company interview for software eng" has been consistently a cursed problem in the industry. It's not actually a mystery. The problem is that that sort of test is de facto illegal (in the US) due to the "4/5ths rule".
Uehreka 2y ago
If you create a standardized test it will be gamed. Even with the small modicum of standardization around interview questions that we currently see, people have published books like Cracking The Code Interview, making it easier for people who don’t have the skills for a particular job to pass interviews at any place that uses standard-ish questions. Furthermore, as an avowed enemy of “Clean Code”, I don’t want to see standardization because I fear that well-promoted ideas that I think are terrible would become required dogma. I prefer chaos over order that I don’t like.
- gwbas1c 2y ago
  > If you create a standardized test it will be gamed. Well, the medial profession has a standardized licensing process. It's not perfect, but it certainly keeps the interview process to (mostly) mutual interest. I think we can learn from the medical profession here. Otherwise, "I prefer chaos" implies that the incompetents are the ones who are the ones who will lose.
- herval 2y ago
  Why do standardized tests work for so many other industries?
- 8f2ab37a-ed6c 2y ago
  How do we let someone fly hundreds of people through the upper atmosphere with a certificate, but you can't make a login page with javascript without a unique multi-day interview for each distinct company?
  - supernewton 2y ago
    For air transport in the U.S., it's not just one certificate, it's many. You get your private license, instrument rating, multi-engine rating, commercial certificate, instructor certificate, and finally the air transport certificate. And you're not allowed to even think about that last step until you've accumulated 1500 flight hours on the previous steps. Being allowed to write a Javascript login page is easy pickings compared to that.
  - 2y ago
jandrewrogers 2y ago
For many roles the interview is as much a cognitive function and socialization test as it is a skills test. You can have exquisitely detailed knowledge of systems internals (skill) but if you have limited working memory (cognitive) then you will struggle to design non-trivial systems software. These are orthogonal dimensions. You might prefer someone with high cognitive aptitude and low skill, since the latter is readily trainable. Cataloging a list of skills is insufficient information to determine if a person is suited for a role. I don't find it likely that software engineers will be subjecting themselves to a battery of standardize cognitive function tests any time soon.
marcinzm 2y ago
> Unlike airline pilots (or I'm certain many other professions), every company in the valley insists on re-interviewing a candidate in their own custom and unique way, instead of trusting some sort of an industry-wide certificate only achievable through a standardized test. Wonder if this will ever be solved. The airplane pilot interview process on top of the standardized government certifications includes: - On-line application (resume and cover letter) - On-line Psychometric/Aptitude testing (sometimes this is hands-on, on site for some airlines) - Video Interview, SKYPE or Telephone interview - Assessment Day (includes: Technical Questions / Panel Behavioral Interview / Scenario Based Questions / Flight Planning Exercise and sometimes a Clinical Assessment) - Sim Test - Referee Check - Medical Check The exact details differ by airline and I'm assuming the risk profile of the cargo (ie: passengers or not). Gosh, not so different from software engineers, is it? Except you also need to do a bunch of bureaucratic certifications on top of that.
- wan23 2y ago
  It's completely different from software engineering interviews. The process you described for airline pilots gets to the actual qualifications for the job. Whereas for software engineers, literally no one needs to reverse a binary tree yet they base the decision on in large part on this sort of question. Ideally, the BS would be encapsulated in a certification so that interviews can focus on the real useful stuff.
  - marcinzm 2y ago
    So needing to get yearly re-certifications in Leetcode questions is viewed as preferably by you than having to only study them for interviews? And spending 10+ hours on coding exercises, un-paid, per job is also seen as preferably by you to the current status on top of the yearly Leetcode certification? On top of a full panel of behavioral interviews? Plus needing to take an IQ style assessment online before each job (see the second item on my list)? I'll take having to study Leetcode every few years over all of that any day of the week.
- Aurornis 2y ago
  Not to mention all of the licensing, regulations, and formalized training hours that you have to put in just to reach that point. It’s all substantially harder than studying LeetCode for a short slice of your life. It’s amazing how often I hear about how easy interviews are in other professions, according to engineers who dislike coding interviews. Then you look into those other professions and it turns out changing jobs is actually a lot harder than the internet comments would lead you to think.
neilv 2y ago
Airline pilot is not a great choice for comparison. Airline pilots are selected to be entrusted with many lives, with the utmost professionalism, and to perform with ability under stress. And it shows, in the amazing track record of aviation safety. Most software developers, on the other hand, are mainly paid to mechanically type out code, cribbing from StackOverflow or ChatGPT whenever they don't know something, to "get it done". And it shows, with the atrocious track record of the entire industry at information security, for one obvious metric.
elbasti 2y ago
IMHO, it's because the single most important characteristic for software engineers, at the vast majority of companies, is that they ship features. And that's a skill that's incredibly hard to test for.
ff317 2y ago
The problem is that software development is less like hiring an airline pilot or a structural engineer, and more like hiring an artist. Try making up a "standard exam" that will tell you whether an artist will produce several great unique works for you in the future, so you know which one to hire...
- itronitron 2y ago
  Will they be required to use Scrum and forced to use Jira?
- drewcoo 2y ago
  Yet frankly, what most of us do is more like plumbing than art. In that we're just fitting systems together and in that it's actual skilled labor and in that we're seen by everyone else as the ones willing to do the shitty work. Management puts up with us and they pay us because even though they think they can do our work, they wouldn't want to. Plumbers are licensed and unionized, two possible solutions to the problems posed in this thread.
- 8f2ab37a-ed6c 2y ago
  That's an interesting point, but then one wonders that if software eng are ultimately artists, why are we not having them work on their portfolios like the other art disciplines? Is that the fundamental problem?
  - gwbas1c 2y ago
    > one wonders that if software eng are ultimately artists I find treating software engineering like art is a very dangerous approach and bound for failure. A lot of software engineering comes down to being comfortable with code and how the computer works. That has little to nothing to do with art.
    - recursive 2y ago
      A lot of painting still life comes down being comfortable with brushes and knowing how paint works. A lot of music comes down to know scales and chords work. Most art still requires fundamental mechanics.
itronitron 2y ago
The fundamental truth is that companies that want to hire people do hire people. There are many more job positions posted than will ever be hired for, despite the availability of suitable talent.

gjsman-1000 2y ago

Part of me wonders if the recruiting itself is over-engineered anyway. I mean, imagine if you just asked: Implement Bubble Sort, in 2 Languages of your choice, with multithreading or other language-provided parallelism, with the correct number of parallel threads to be most algorithmically efficient Would that really not weed out a lot of people? I think it would. I know the above algorithm is hardly production-ready, but the requirements are easy to understand. (It's also a bit of a trick question - there is no optimal algorithmically efficient number of threads in a bubble sort, only the number of CPU cores in the system.)

anonymoushn 2y ago
What's your approach for parallelizing bubble sort in a way that profits from cores beyond the first?
- thdc 2y ago
  It's probably alternating the comparisons. Compare [0, 1] [2, 3] [4, 5] ... in parallel and swap if necessary, compare/swap [1, 2] [3, 4] [5, 6] ... in parallel, then go back and forth until no more swaps are made - second element in pair is always greater/less than the first. That does suggest that the theoretical ideal number of threads is n / 2 ignoring cores, though you'll also want to consider things like cache line size, cache coherency, and actual problem size so it's probably less. At the end of the day, the important thing would be to actually benchmark your attempts and seeing how it scales with processors/problem size and checking the isoefficiency. I think it was a bad question.
aidos 2y ago
What are you trying to achieve here? Are you looking to remove a bunch of competent developers by asking weird trick questions?
- gjsman-1000 2y ago
  1. It's the dumbest algorithm for sorting possible (bubble sort). Compare two objects, swap them if one is bigger than the other, go down the list, repeat. If a developer doesn't know that algorithm, what algorithm could they possibly know? It's the lowest bar. 2. 2 Languages of your choice is enough to show you aren't a frameworker and can think in more than one box. Doesn't matter if you do it in JavaScript and C#, Go and Rust, Python and Haskell. It's a chance to show off, while being quite obtainable for a competent developer. 3. The parallelism trick question merely shows that you actually understand what you are talking about. A leetcoder might make the trap of assuming it's log(n), or the square root of n elements, or some other overly-thought-through math that is bogus. If you think it through though, bubble sort is simple enough, that it's very easy to realize (in my opinion) why more threads than CPU cores doesn't really help.
  - probably_wrong 2y ago
    > or some other overly-thought-through math that is bogus. So the mathematically-correct answer would be a minus while "in my opinion" would be a plus? I think your interview question brings you to your intended answer: the reason why every company makes their own process is because what's good for you would probably get you an instant fail in NASA and would get you labeled as "experience not relevant" in some research settings.
    - gjsman-1000 2y ago
      > So the mathematically-correct answer would be a minus while "in my opinion" would be a plus? My opinion was only that it would be relatively easy to figure out. As far as I know, it is a certainty that sectioning a bubble sort, more or less than CPU cores, is less efficient. 16 threads on an 8 core CPU just means balancing 2 threads on every CPU core, each doing their own sorting and needing eventual merging back together. There's no way that can be more efficient.
rachofsunshine 2y ago
Our coding problem is easier than that (by a fair margin, it's all completely synchronous single-threaded procedural code unless you're doing something extremely weird) and it weeds out the vast majority of applicants. The same was true of all three of the standard coding problems Triplebyte used. They're not quite literal fizzbuzz, but they require - at best - some basic critical thinking and basic language features, and that is a filter that eliminates 90+% of applicants to dev jobs. Now, granted, this is under time pressure. I imagine, given several hours, most could finish it (although maybe even that is overestimating things). But still. There's an old Randall Munroe article quoting a physicist: > The physicist who mentioned this problem to me told me his rule of thumb for estimating supernova-related numbers: However big you think supernovae are, they're bigger than that. and I feel like this applies to recruiting: however bad you think the average applicant is, they're worse than that.
- josephg 2y ago
  Triplebyte used to send candidates a link to a page which basically listed everything that was in the interview. Most candidates didn’t read it. They could probably have done away with the interview entirely and just put a link at the bottom of the prep material saying “click here to pass the interview”. If anything I think it would have lowered the triplebyte passing rate.
  - judahmeek 2y ago
    Why not try this as an experimental filter then and see what sort of population pass?
    - josephg 2y ago
      Well, triplebyte is dead. That’s one reason not to do the experiment. Also arguably the “customer” for triplebyte was the company that eventually hires the candidate. They’re paying in part for the screening process triplebyte did. We wouldn’t have been doing our job if we skipped the interview part of the process.
- 2y ago
zeroCalories 2y ago
I find picking good questions is hard, and many fall into similar patterns, making them something candidates can practice for. Even your question isn't something I'd necessarily ask on the spot. Many engineers don't use parallelism in their day-to-day work(webdevs). The part about making it efficient is interesting, but feels borderline like a trick question that a good engineer could fumble.
- gjsman-1000 2y ago
  > Even your question isn't something I'd necessarily ask on the spot. Many engineers don't use parallelism in their day-to-day work(webdevs). The part about making it efficient is interesting, but feels borderline like a trick question that a good engineer could fumble. True, it's more of a backend role question. The reason I threw it in there, is from my assumption a leetcode grinder would be very likely to immediately go, "well, the most efficient number of threads is log (n)" or "the most efficient number of threads is the square root of n" or some other plausible-sounding BS answer. But the reason I chose bubble sort, is that it's so simple to understand, that you can fairly easily (I would hope) figure out there's no benefits to more threads than CPU cores at all, as long as you stop and actually think about what it is doing.
gamepsys 2y ago
I don't think we are over-engineering it. You want to "weed out" everyone but the best candidate for the role, or the best candidates for your open roles. It's a very hard problem to identify the "best" person from a group of people. It would be different if all programmers that are good enough are the same, but we all know the skill ceiling on programming is very high. Selecting the best devs is a critical function for any software project.
LargeWu 2y ago
It's also a mostly useless problem for determining engineer quality, in many cases. It tests for pure coding ability, when most organizations should be optimizing for engineers that are going to be productive-if-not-spectacular, that can design and build maintainable systems. Could I have written the above problem back in my engineering days? Probably not, since I went years not working with threads. But I also wasn't working on problems that would ever have benefited from knowing that. Most software engineering roles are essentially building CRUD or ETL systems, maybe with a user interface. Any coding problems given should be optimized for weeding out bozos (which are still plentiful), not for weeding out the most people.
Uehreka 2y ago
I do a tree-search-ish thing when interviewing people. I’ll start with a super basic question about something beginner-ish on their resume. If they can’t answer that, the interview is politely wrapped up. I’ve eliminated a surprising number of people who had jQuery on their resume by asking them to write code that will make a div with the ID “mydiv” disappear if the user clicks a button with the id “mybutton”. After that I ask a super difficult or niche trivia question like “in CSS, what two overflow properties cannot be used together?” If I’m hiring a mid-level frontend developer and they nail that one, I go “fantastic, great answer, do you have any questions for us?” And the interview can end early. But if they miss that, no sweat, I’ll start asking more mid-level technical questions to figure out where they’re at.
timfsu 2y ago
This sounds like what a lot of companies do - except the scaled problem with this approach (and the certification approach of the grandparent) is that most companies want to avoid candidates who've memorized a specific solution, as then they don't get any data about whether they can code anything aside from what was memorized. The other problem is that implementing bubble sort will tell you about their skills in a particular dimension, but being a software engineer these days may look very different depending on the job.

dataminded 2y ago

It's because companies don't want capable, experienced or well-equipped. They want genius and it is really hard to test for genius. Granted, almost nobody that gets through any process is an actual genius....

onlyrealcuzzo 2y ago
It's because you have to go through a lengthy process to become a pilot - and tech companies want to be able to hire anyone regardless of training. Combine that with the fact that the upper bound on pay for SWEs is considerably higher than pilots...
debacle 2y ago
I have never needed to hire a genius in the last 12 years. In fact, often times I've had to pass on candidates that were overqualified.
duped 2y ago
I'd say it's the exact opposite. There are hordes of unqualified people applying to every software dev role imaginable regardless of what you put in the job description or requirements. The tests are there because people are good at lying but bad at faking skills.
- rachofsunshine 2y ago
  Seriously, if you've never hired before, you have no idea how bad this can get. Here's [1] our practice coding problem. It's quite similar to the one we use on our interview, and not too far from the one Triplebyte used in the past (ours is tuned to be slightly harder at the beginning and slightly easier at the end). The vast majority of candidates, even with some reasonable pre-filtering, do not get past the first step. A very non-trivial number would not even get that far. [1] https://www.otherbranch.com/practice-coding-problem https://www.otherbranch.com/practice-coding-problem
  - itronitron 2y ago
    That is the sort of coding problem I would love to work on as an exercise however the 25 minute time limit would put me off and I wouldn't even start. I enjoy programming and I don't like to feel rushed, and if you're placing that sort of time constraint then you probably aren't a great company to work for.

listenallyall 2y ago

A standardized test exists for lawyers (two, if you count the LSATs) but law firms still interview potential hires.

saulpw 2y ago
Do these firms give them their equivalent of an LSAT or the bar exam? AFAIK it's a "non-technical" interview.
janalsncm 2y ago
Is law a good example? My understanding is if you didn’t go to a top 14 school (whoever came up with that arbitrary number) it basically forecloses on the best opportunities.
- evantbyrne 2y ago
  A similar pattern exists in tech startup hiring practices and which ones attract VC funding. Not unusual to see funded startups with founders who have no work experience but Stanford degrees. Before my time at The Atlantic, I had a couple recruiters for no-name startups tell me I didn't have a prestigious enough background to be hired. There is a highly-visible class hierarchy in tech that many people in the industry seem unaware of. Perhaps this is because base salaries are high-enough that the middle class is just happy to be included at all.
  - happyopossum 2y ago
    Don’t listen to recruiters? They lie. I have been at several startups, including a massive unicorn, and now work at a faang. I have no college experience whatsoever.
  - rachofsunshine 2y ago
    100%. I'm not sure if this was always the case or if it was a slow result of tech becoming overrun by finance, but it's a very motivating thing for me. I started a company for a lot of reasons, and this isn't the top one, but I sure would love to show success without playing the class-signalling games that the valley seems overrun with. On the other hand, I'm posting my content post here in part because I know the HN candidate pool is about a trillion times better than I'll get anywhere else. So perhaps I've already lost that battle.

sevensor 2y ago

This process is also pretty much guaranteed never to yield mid-career geniuses at the height of their powers. Those candidates don't go looking for work at all. Work comes looking for them. Why would they go on _any_ jobs platform, ever? Effective filtering of the candidates who actually engage with the platform can, at best, accurately identify the next tier down: effective engineers in mid- and late-career, and inexperienced whiz kids. Not that this is a bad thing; that first category makes the world go 'round.

josephg 2y ago
> Those candidates don't go looking for work at all. Work comes looking for them. Why would they go on _any_ jobs platform, ever? Because I don't know what’s out there, or who will give me the best offer. If you’re skilled and in the middle of your career, it’s easy to find a job, but if your options are wide open, a matchmaking service like this with a wide pool of companies is very valuable.

nordsieck 2y ago

> It's interesting to see how much "standardized cross-company interview for software eng" has been consistently a cursed problem in the industry. It's not actually a mystery. The problem is that that sort of test is de facto illegal (in the US) due to the "4/5ths rule".

Uehreka 2y ago

If you create a standardized test it will be gamed. Even with the small modicum of standardization around interview questions that we currently see, people have published books like Cracking The Code Interview, making it easier for people who don’t have the skills for a particular job to pass interviews at any place that uses standard-ish questions. Furthermore, as an avowed enemy of “Clean Code”, I don’t want to see standardization because I fear that well-promoted ideas that I think are terrible would become required dogma. I prefer chaos over order that I don’t like.

gwbas1c 2y ago
> If you create a standardized test it will be gamed. Well, the medial profession has a standardized licensing process. It's not perfect, but it certainly keeps the interview process to (mostly) mutual interest. I think we can learn from the medical profession here. Otherwise, "I prefer chaos" implies that the incompetents are the ones who are the ones who will lose.
herval 2y ago
Why do standardized tests work for so many other industries?
8f2ab37a-ed6c 2y ago
How do we let someone fly hundreds of people through the upper atmosphere with a certificate, but you can't make a login page with javascript without a unique multi-day interview for each distinct company?
- supernewton 2y ago
  For air transport in the U.S., it's not just one certificate, it's many. You get your private license, instrument rating, multi-engine rating, commercial certificate, instructor certificate, and finally the air transport certificate. And you're not allowed to even think about that last step until you've accumulated 1500 flight hours on the previous steps. Being allowed to write a Javascript login page is easy pickings compared to that.
- 2y ago

jandrewrogers 2y ago

For many roles the interview is as much a cognitive function and socialization test as it is a skills test. You can have exquisitely detailed knowledge of systems internals (skill) but if you have limited working memory (cognitive) then you will struggle to design non-trivial systems software. These are orthogonal dimensions. You might prefer someone with high cognitive aptitude and low skill, since the latter is readily trainable. Cataloging a list of skills is insufficient information to determine if a person is suited for a role. I don't find it likely that software engineers will be subjecting themselves to a battery of standardize cognitive function tests any time soon.

marcinzm 2y ago

> Unlike airline pilots (or I'm certain many other professions), every company in the valley insists on re-interviewing a candidate in their own custom and unique way, instead of trusting some sort of an industry-wide certificate only achievable through a standardized test. Wonder if this will ever be solved. The airplane pilot interview process on top of the standardized government certifications includes: - On-line application (resume and cover letter) - On-line Psychometric/Aptitude testing (sometimes this is hands-on, on site for some airlines) - Video Interview, SKYPE or Telephone interview - Assessment Day (includes: Technical Questions / Panel Behavioral Interview / Scenario Based Questions / Flight Planning Exercise and sometimes a Clinical Assessment) - Sim Test - Referee Check - Medical Check The exact details differ by airline and I'm assuming the risk profile of the cargo (ie: passengers or not). Gosh, not so different from software engineers, is it? Except you also need to do a bunch of bureaucratic certifications on top of that.

wan23 2y ago
It's completely different from software engineering interviews. The process you described for airline pilots gets to the actual qualifications for the job. Whereas for software engineers, literally no one needs to reverse a binary tree yet they base the decision on in large part on this sort of question. Ideally, the BS would be encapsulated in a certification so that interviews can focus on the real useful stuff.
- marcinzm 2y ago
  So needing to get yearly re-certifications in Leetcode questions is viewed as preferably by you than having to only study them for interviews? And spending 10+ hours on coding exercises, un-paid, per job is also seen as preferably by you to the current status on top of the yearly Leetcode certification? On top of a full panel of behavioral interviews? Plus needing to take an IQ style assessment online before each job (see the second item on my list)? I'll take having to study Leetcode every few years over all of that any day of the week.
Aurornis 2y ago
Not to mention all of the licensing, regulations, and formalized training hours that you have to put in just to reach that point. It’s all substantially harder than studying LeetCode for a short slice of your life. It’s amazing how often I hear about how easy interviews are in other professions, according to engineers who dislike coding interviews. Then you look into those other professions and it turns out changing jobs is actually a lot harder than the internet comments would lead you to think.

neilv 2y ago

Airline pilot is not a great choice for comparison. Airline pilots are selected to be entrusted with many lives, with the utmost professionalism, and to perform with ability under stress. And it shows, in the amazing track record of aviation safety. Most software developers, on the other hand, are mainly paid to mechanically type out code, cribbing from StackOverflow or ChatGPT whenever they don't know something, to "get it done". And it shows, with the atrocious track record of the entire industry at information security, for one obvious metric.

elbasti 2y ago

IMHO, it's because the single most important characteristic for software engineers, at the vast majority of companies, is that they ship features. And that's a skill that's incredibly hard to test for.

ff317 2y ago

The problem is that software development is less like hiring an airline pilot or a structural engineer, and more like hiring an artist. Try making up a "standard exam" that will tell you whether an artist will produce several great unique works for you in the future, so you know which one to hire...

itronitron 2y ago
Will they be required to use Scrum and forced to use Jira?
drewcoo 2y ago
Yet frankly, what most of us do is more like plumbing than art. In that we're just fitting systems together and in that it's actual skilled labor and in that we're seen by everyone else as the ones willing to do the shitty work. Management puts up with us and they pay us because even though they think they can do our work, they wouldn't want to. Plumbers are licensed and unionized, two possible solutions to the problems posed in this thread.
8f2ab37a-ed6c 2y ago
That's an interesting point, but then one wonders that if software eng are ultimately artists, why are we not having them work on their portfolios like the other art disciplines? Is that the fundamental problem?
- gwbas1c 2y ago
  > one wonders that if software eng are ultimately artists I find treating software engineering like art is a very dangerous approach and bound for failure. A lot of software engineering comes down to being comfortable with code and how the computer works. That has little to nothing to do with art.
  - recursive 2y ago
    A lot of painting still life comes down being comfortable with brushes and knowing how paint works. A lot of music comes down to know scales and chords work. Most art still requires fundamental mechanics.

itronitron 2y ago

The fundamental truth is that companies that want to hire people do hire people. There are many more job positions posted than will ever be hired for, despite the availability of suitable talent.