3 ms·

Would be interesting to hear Rachel's take on all the contract recruiting (?) firms that claim to do the same thing, test candidates and then provide them to em

Would be interesting to hear Rachel's take on all the contract recruiting (?) firms that claim to do the same thing, test candidates and then provide them to employers- just on remote contracts for temp work. The OG in this space is Toptal, but there seems to be a lot of VC money in a bunch of upstarts in this space. They all say 'we screen our developers through an extremely rigorous yada yada process'. They tend to have a lot of developing world talent. Not sure if that's the future of online talent platforms or what have you

rachofsunshine 2y ago
I don't have special knowledge of Toptal's screening process. But when I look at their pitch (linked below), what sticks out at me is the lack of specifics. ----- (From https://www.toptal.com/top-3-percent https://www.toptal.com/top-3-percent) > We also test each applicant's technical knowledge and problem-solving ability through various assessments. "Various assessments" is about as maximally vague as you can be. Imagine if I asked you how you'd build a back end and you went "oh, you know, various technologies, um, APIs, probably some functions" - that's the level we're operating on here. Here's how you do it not vaguely: Otherbranch's assessment is, currently, a 90 minute call with an engineer. It's synchronous over video and screen-share and split into 3 subsections: coding, knowledge, and system design. Coding is a small app in the console, you can find an example problem at https://www.otherbranch.com/practice-coding-problem https://www.otherbranch.com/practice-coding-problem . Knowledge is a mix of CS/algos, full-stack web development, and "deep tech" (systems-level/security/internals). System design is really standard "build this basic business system and scale it up". If you want more depth on our grading, I can show you that, although I don't share our actual questions verbatim. I did the structure and I've led large-scale assessments before, and our interviewers have done thousands of hours of interviews and several are ex-FAANG. Now, granted, we're in one domain and they're in many, so specifics may be hard to produce here. (And they may even be bad sales practice, so there might be non-malicious reasons for not including them.) But it still catches my eye. > we typically only advance candidates with exceptional results in this phase. 26.4% of candidates pass basic English screening, according to the numbers next to the previous claim. Of those, they claim, 7.4% pass this step. Note that no screenings for actual skills have occurred prior to this step. If we assume that domain skill and English screening are weakly correlated (this is probably a bad assumption, literally any two skills anywhere have positive correlations), they're passing roughly 25% of applicants by skill. Let's generously say 15-20%. And the 80-85th percentile of applicant is horrible. So I just don't buy "exceptional results" here. > Each candidate is interviewed by Toptal screeners who are experts in their functional domain. Again, vague. Interviewed for how long? With what criteria? How are you determining that they are "experts in their functional domain"? > Each candidate is assigned a test project to evaluate whether they can "walk the walk." Test projects take 1-3 weeks are comprehensive and provide real-world scenarios for candidates to demonstrate their competence, thoroughness, professionalism, and integrity. This part looks better! Until you look at the numbers next to it, which indicate a 90+% pass rate for this phase (and still a roughly 10% pass rate from "people who cleared basic English language skills". It's easy to say you're rigorously assessing people. But when they're so evasive about the details, I get suspicious. ----- I'm not saying Toptal isn't better than nothing. I'm not an expert on international hiring, and they seem to be doing pretty well as a business. And this vagueness may very well be a feature, not a bug - my verbosity gets me into trouble sometimes when I write copy. In fact, this very post demonstrates a good reason NOT to be specific, because armchair experts will pick your exact numbers apart without the appropriate context. (I justify myself by saying that this is, in fact, my area of expertise.) But if you want my snap take, there it is. Toptal people reading this thread, feel free to drop in with details if you got em!

rachofsunshine 2y ago

I don't have special knowledge of Toptal's screening process. But when I look at their pitch (linked below), what sticks out at me is the lack of specifics. ----- (From https://www.toptal.com/top-3-percent https://www.toptal.com/top-3-percent) > We also test each applicant's technical knowledge and problem-solving ability through various assessments. "Various assessments" is about as maximally vague as you can be. Imagine if I asked you how you'd build a back end and you went "oh, you know, various technologies, um, APIs, probably some functions" - that's the level we're operating on here. Here's how you do it not vaguely: Otherbranch's assessment is, currently, a 90 minute call with an engineer. It's synchronous over video and screen-share and split into 3 subsections: coding, knowledge, and system design. Coding is a small app in the console, you can find an example problem at https://www.otherbranch.com/practice-coding-problem https://www.otherbranch.com/practice-coding-problem . Knowledge is a mix of CS/algos, full-stack web development, and "deep tech" (systems-level/security/internals). System design is really standard "build this basic business system and scale it up". If you want more depth on our grading, I can show you that, although I don't share our actual questions verbatim. I did the structure and I've led large-scale assessments before, and our interviewers have done thousands of hours of interviews and several are ex-FAANG. Now, granted, we're in one domain and they're in many, so specifics may be hard to produce here. (And they may even be bad sales practice, so there might be non-malicious reasons for not including them.) But it still catches my eye. > we typically only advance candidates with exceptional results in this phase. 26.4% of candidates pass basic English screening, according to the numbers next to the previous claim. Of those, they claim, 7.4% pass this step. Note that no screenings for actual skills have occurred prior to this step. If we assume that domain skill and English screening are weakly correlated (this is probably a bad assumption, literally any two skills anywhere have positive correlations), they're passing roughly 25% of applicants by skill. Let's generously say 15-20%. And the 80-85th percentile of applicant is horrible. So I just don't buy "exceptional results" here. > Each candidate is interviewed by Toptal screeners who are experts in their functional domain. Again, vague. Interviewed for how long? With what criteria? How are you determining that they are "experts in their functional domain"? > Each candidate is assigned a test project to evaluate whether they can "walk the walk." Test projects take 1-3 weeks are comprehensive and provide real-world scenarios for candidates to demonstrate their competence, thoroughness, professionalism, and integrity. This part looks better! Until you look at the numbers next to it, which indicate a 90+% pass rate for this phase (and still a roughly 10% pass rate from "people who cleared basic English language skills". It's easy to say you're rigorously assessing people. But when they're so evasive about the details, I get suspicious. ----- I'm not saying Toptal isn't better than nothing. I'm not an expert on international hiring, and they seem to be doing pretty well as a business. And this vagueness may very well be a feature, not a bug - my verbosity gets me into trouble sometimes when I write copy. In fact, this very post demonstrates a good reason NOT to be specific, because armchair experts will pick your exact numbers apart without the appropriate context. (I justify myself by saying that this is, in fact, my area of expertise.) But if you want my snap take, there it is. Toptal people reading this thread, feel free to drop in with details if you got em!