If you want generalists, you won't really be able to test them properly by asking questions at random: All you are hoping for is that they share the same backg

If you want generalists, you won't really be able to test them properly by asking questions at random: All you are hoping for is that they share the same background as you do. The moment it's you asking the question, and they can't really check sources, you have already lost. As far as hardcore computer science abilities, a test still won't help, because what you are really testing is how far people are from college: Generalist cs is touched in college, but there are entire sections of computer science that someone with 10 years of experience might not have had to touch. I've had to do pretty specific CS stuff for work, but to get any of that done, I started with weeks of research, and then add my own spin to our specific situation top. You can't test the ability to do that by just checking if someone has memorized Raft, or black red trees, or any other random problem. I don't have a lot of CS algorithms memorized: that's why there's the internet, and why I have Knuth in my shelf. You want to check for hard cs, ask the candidate to choose his favorite CS topic, and ask for am impromptu presentation on that. anything else is just going to fail at finding candidates that have different backgrounds than you, and that's PRECISELY the people you want to hire.