I have a similar perspective in that I think most engineers probably could get hired, but there's enough randomness in the interview process that we probably wo

I have a similar perspective in that I think most engineers probably could get hired, but there's enough randomness in the interview process that we probably wouldn't. Not just in who gets advanced to the phone screen, but also variability in test performance. Most of the interviews were open-ended and had multiple correct answers - but it still really only answers, "did the candidate think of the right answer in X minutes". One of our questions pretty much explicitly required knowledge of how to implement a rolling hash to pass, and that's not something I knew when I worked at the company. I'm of the mind that most interview questions can only reliably identify people who are not capable of coding, and roughly gauge the algorithms knowledge. I had floated the idea of making interview experiences that more closely resemble work, like having the candidates make PRs in a mock repository. Better yet we could see not just how they make a PR but also review PRs and make comments. But often the primary concern is logistical. It's easy to train people up on 1-hour problems with a grading rubric, than a longer more open ended interview.