4 ms·

This feels like a solution in search of a problem Fact: The LeetCode loop sucks. So couldn't any Director or VP in big tech just decide they are done giving th

by shmatt 5y ago

This feels like a solution in search of a problem Fact: The LeetCode loop sucks. So couldn't any Director or VP in big tech just decide they are done giving that interview? They have the resources to spin up something like this in a week. Even outside big tech, companies can very well think of non LeetCode style interviews, some already do The problem is, many engineers want new engineers forced into this type of interview, because they had to do it, so you have to do it too. Not for any reason better than that I'm not sure a full featured well polished product like this one can change toxic engineering culture when it comes to joining the club

sokoloff 5y ago
I think there’s (unfortunately) a need for a low-effort high-pass filter for candidates because there are so many utterly unqualified candidates chasing the dollars that a software job can bring. I don’t blame them, but I also think the bottom 10% of all potential programmers is way over-represented in the applicant pool and finding a way to filter them out effectively has value. It doesn’t have to be leetcode, but it does have to not be “have an engineer spend an hour with everyone”
- garyjlin 5y ago
  This is what we saw too when we were hiring for our own engineers 2 months ago. A surprisingly good filter is "can you even get started?". Like, the fact that our interviews required users to navigate Github, setup a dev env (even if it's just the make command), understand the existing code and prompt alone filtered out a non-negligible percentage of the candidate pool.
  - sokoloff 5y ago
    That is simultaneously crazy and entirely believable from what I’ve seen. “Experienced” Java devs who couldn’t install Java or set $JAVA_HOME, devs who couldn’t use git to checkout code, and other “how did you possibly get to this point in the overall space?”
ketzo 5y ago
> They have the resources to spin up something like this in a week That sentence right there is a pretty solid indicator that it’s a solid business to be in, no? What VP wants to take engineer time to create an internal tool when they could pay $40/user/month or whatever? How many SaaS startups do we see right now that occupy that exact sentiment area? As far as this being more of a culture change than a process change — well, good products can change culture. Making something much easier is a huge step on the way to getting people to do it.
- garyjlin 5y ago
  Completely agree! This is indeed a big problem, and we have so many CTOs, VPs and founders willing to jump into 6 figure contracts if it works. The biggest risk here is execution, not market size.
- JimDabell 5y ago
  > What VP wants to take engineer time to create an internal tool when they could pay $40/user/month or whatever? The cheapest pricing for this is $600/mo, the next plan up is $2,400/mo, the enterprise plans start at $60k/mo.
  - ketzo 5y ago
    I mean, how many engineering interviews is a big enterprise company running per month? 1000 feels like a comfortable lower bound. Most of those would use this tool at least twice during their interview process. So you're paying... $60k/2000 = a cost of $30 per interview. If you save your company half an hour of engineer time (at $60/hr, another comfortable lower bound) you have already made back the cost. And that's just strictly cost reduction. If this is genuinely a better interview process, if you get more applicants and qualified candidates as a result of using the tool, of course the value of said tool is gonna be even higher. I'm pulling numbers out of a hat, obviously, but I don't think any of this is at all unreasonable. And once you start putting it like this, I think you can start to see the justification for that kind of pricing. Edit: well, just looking at their Enterprise pricing in a little more detail, they're pricing at 60k/200 = $300/interview, then $30/interview after that. That's... a lot steeper, and a lot less friendly to mid-tier companies. But hey, frankly, I can still see the justification for it -- if it's really good.
garyjlin 5y ago
Right, biggest pain point we're trying to solve for is engineers getting pulled away from their work to build, maintain, conduct, and evaluate interviews. I recently spoke to Stripe engineers that spend 2 hours a day, every day, on this. A Brex EM also every engineer on the team conducts 1 - 2 onsite interviews every week. Eng time is the biggest expense in the recruiting funnel, and it's particularly difficult to solve for that problem. We're hoping that if Litebulb can take care of some of the more tedious code analysis work, then we can help teams reduce their eng spend, or use the newly available time to build deeper connections with candidates as humans. When it comes to changing engineering culture, I completely agree that a single tool like this isn't going to cut it, but we're seeing a big shift in new startups focusing a lot more on real work relevant take homes and/or work trials. For example, Gumroad (disclaimer: a client), does a Litebulb interview as the entrypoint, then it's straight to a 4-8 week paid trial on flexible hours with a choice of which project to work on. We're hoping that we can be a part of, and accelerate, this culture shift.
- hemloc_io 5y ago
  Hmmm, maybe not a question for you, but for Gumroad, isn't there already an issue with candidates investing so much time in the process? (E.g. with any large company it can literally take 3 months+) For me as someone who just finished interviewing, if you told me that I either have to work on two jobs for 4-8 weeks or quit my current job for a limited contract that may or may not work out I'd almost certainly say no. (I think most people would too, I mean people already complain about take home quizes etc.)
Melatonic 5y ago
I also know a lot of Directors who like buying a solid product with solid support. That could be a big selling point on its own.
justshowpost 5y ago
> The problem is, many engineers want new engineers forced into this type of interview, because they had to do it, so you have to do it too. Not for any reason better than that I doubt HR people are on a coordinated mission to revenge their own suffering by letting new candidates suffer the same horrible LeetCode fate. Instead, I rather think LeetCode is chosen because it’s the easiest way, the path of least effort, or laziness.

by shmatt 5y ago

sokoloff 5y ago

I think there’s (unfortunately) a need for a low-effort high-pass filter for candidates because there are so many utterly unqualified candidates chasing the dollars that a software job can bring. I don’t blame them, but I also think the bottom 10% of all potential programmers is way over-represented in the applicant pool and finding a way to filter them out effectively has value. It doesn’t have to be leetcode, but it does have to not be “have an engineer spend an hour with everyone”

garyjlin 5y ago
This is what we saw too when we were hiring for our own engineers 2 months ago. A surprisingly good filter is "can you even get started?". Like, the fact that our interviews required users to navigate Github, setup a dev env (even if it's just the make command), understand the existing code and prompt alone filtered out a non-negligible percentage of the candidate pool.
- sokoloff 5y ago
  That is simultaneously crazy and entirely believable from what I’ve seen. “Experienced” Java devs who couldn’t install Java or set $JAVA_HOME, devs who couldn’t use git to checkout code, and other “how did you possibly get to this point in the overall space?”

ketzo 5y ago

> They have the resources to spin up something like this in a week That sentence right there is a pretty solid indicator that it’s a solid business to be in, no? What VP wants to take engineer time to create an internal tool when they could pay $40/user/month or whatever? How many SaaS startups do we see right now that occupy that exact sentiment area? As far as this being more of a culture change than a process change — well, good products can change culture. Making something much easier is a huge step on the way to getting people to do it.

garyjlin 5y ago
Completely agree! This is indeed a big problem, and we have so many CTOs, VPs and founders willing to jump into 6 figure contracts if it works. The biggest risk here is execution, not market size.
JimDabell 5y ago
> What VP wants to take engineer time to create an internal tool when they could pay $40/user/month or whatever? The cheapest pricing for this is $600/mo, the next plan up is $2,400/mo, the enterprise plans start at $60k/mo.
- ketzo 5y ago
  I mean, how many engineering interviews is a big enterprise company running per month? 1000 feels like a comfortable lower bound. Most of those would use this tool at least twice during their interview process. So you're paying... $60k/2000 = a cost of $30 per interview. If you save your company half an hour of engineer time (at $60/hr, another comfortable lower bound) you have already made back the cost. And that's just strictly cost reduction. If this is genuinely a better interview process, if you get more applicants and qualified candidates as a result of using the tool, of course the value of said tool is gonna be even higher. I'm pulling numbers out of a hat, obviously, but I don't think any of this is at all unreasonable. And once you start putting it like this, I think you can start to see the justification for that kind of pricing. Edit: well, just looking at their Enterprise pricing in a little more detail, they're pricing at 60k/200 = $300/interview, then $30/interview after that. That's... a lot steeper, and a lot less friendly to mid-tier companies. But hey, frankly, I can still see the justification for it -- if it's really good.

garyjlin 5y ago

Right, biggest pain point we're trying to solve for is engineers getting pulled away from their work to build, maintain, conduct, and evaluate interviews. I recently spoke to Stripe engineers that spend 2 hours a day, every day, on this. A Brex EM also every engineer on the team conducts 1 - 2 onsite interviews every week. Eng time is the biggest expense in the recruiting funnel, and it's particularly difficult to solve for that problem. We're hoping that if Litebulb can take care of some of the more tedious code analysis work, then we can help teams reduce their eng spend, or use the newly available time to build deeper connections with candidates as humans. When it comes to changing engineering culture, I completely agree that a single tool like this isn't going to cut it, but we're seeing a big shift in new startups focusing a lot more on real work relevant take homes and/or work trials. For example, Gumroad (disclaimer: a client), does a Litebulb interview as the entrypoint, then it's straight to a 4-8 week paid trial on flexible hours with a choice of which project to work on. We're hoping that we can be a part of, and accelerate, this culture shift.

hemloc_io 5y ago
Hmmm, maybe not a question for you, but for Gumroad, isn't there already an issue with candidates investing so much time in the process? (E.g. with any large company it can literally take 3 months+) For me as someone who just finished interviewing, if you told me that I either have to work on two jobs for 4-8 weeks or quit my current job for a limited contract that may or may not work out I'd almost certainly say no. (I think most people would too, I mean people already complain about take home quizes etc.)

Melatonic 5y ago

I also know a lot of Directors who like buying a solid product with solid support. That could be a big selling point on its own.

justshowpost 5y ago

> The problem is, many engineers want new engineers forced into this type of interview, because they had to do it, so you have to do it too. Not for any reason better than that I doubt HR people are on a coordinated mission to revenge their own suffering by letting new candidates suffer the same horrible LeetCode fate. Instead, I rather think LeetCode is chosen because it’s the easiest way, the path of least effort, or laziness.