6 ms·

ChatGPT could probably pass most FAANG programming interviews - it's successfully done linked list inversion, binary tree searches, dynamic programming like kna

ChatGPT could probably pass most FAANG programming interviews - it's successfully done linked list inversion, binary tree searches, dynamic programming like knapsack, and will even do program design if you ask it how to design a data structure or class.

bugfix-66 4y ago
The training set is full of code for these problems. Put your query into Google and see how many thousands of answers appear. Your tests are in the training set.
- dpflan 4y ago
  This idea is why I wanted to see the code created by GPT at each step during this conversation for this post...
- avip 4y ago
  True but the horrifying truth is 95% of our actual problems at work are also in the training set...
  - imtringued 4y ago
    My biggest problems are obscure bugs that I would hardly understand even if the AI would just give me the workaround.
teaearlgraycold 4y ago
Oh wow. I'd love to see someone set up an environment where GPT solves a programming interview incrementally as a human would - with a teleprompter of describe-as-you-go text for the interviewee to read out. Couple it with speech-to-text for follow up questions and you can conman your way through any coding focused interview. Not that I want or need this to pass interviews. But it would finally force the industry to find a better way to evaluate candidates.
- aloner 4y ago
  Cheat on your Technical Interview as a Service™
ninkendo 4y ago
I pasted in the question I typically ask candidates at $dayjob. It did better than the vast majority of candidates. (It’s a very open ended architectural question that’s specific to the industry I’m in.) At the end I said “write it in Rust” and it wrote a plausibly good implementation. I’m not sure I can trust remote interviews any more…
- qzw 4y ago
  Actually it sounds like recruiting just got a lot easier for you!
- abecedarius 4y ago
  That's close to answering a question from a bit over a year ago: "in 5 years will there be an AI that's better than 90% of unassisted working programmers at solving new leetcode-type coding interview questions posed in natural language?" https://news.ycombinator.com/item?id=29020401 https://news.ycombinator.com/item?id=29020401 It didn't get a very welcoming reception -- "highly unlikely", and the tenor of other opinions in those comments then tended the same way. But I was being too conservative. (I thought so at the time but it felt sort of outside an Overton window of reasonable technology opinions.)
  - imtringued 4y ago
    What? Leetcode is purely about memorization. Of course a model trained on those solutions will regurgitate them. That how humans do it!
    - abecedarius 4y ago
      you missed the word 'new'
- naiv 4y ago
  Same here. It passed the tests for backend / frontend and created perfect unit tests as well.
- xrd 4y ago
Der_Einzige 4y ago
It's been an open secret on blind that cheating on remote interviews is highly, highly common, and has been with language models since late 2019. Best part is that these same folks end up getting promoted because as it turns out once you have passed the stupid coding hazing ritual, most jobs at the FAANG will NOT need you to solve leetcode style problems. I'm honestly excited as heck that something will finally kill off this practice. Don't let the door hit you on the ass on the way out, leetcode.

bugfix-66 4y ago

The training set is full of code for these problems. Put your query into Google and see how many thousands of answers appear. Your tests are in the training set.

dpflan 4y ago
This idea is why I wanted to see the code created by GPT at each step during this conversation for this post...
avip 4y ago
True but the horrifying truth is 95% of our actual problems at work are also in the training set...
- imtringued 4y ago
  My biggest problems are obscure bugs that I would hardly understand even if the AI would just give me the workaround.

teaearlgraycold 4y ago

Oh wow. I'd love to see someone set up an environment where GPT solves a programming interview incrementally as a human would - with a teleprompter of describe-as-you-go text for the interviewee to read out. Couple it with speech-to-text for follow up questions and you can conman your way through any coding focused interview. Not that I want or need this to pass interviews. But it would finally force the industry to find a better way to evaluate candidates.

aloner 4y ago
Cheat on your Technical Interview as a Service™

ninkendo 4y ago

I pasted in the question I typically ask candidates at $dayjob. It did better than the vast majority of candidates. (It’s a very open ended architectural question that’s specific to the industry I’m in.) At the end I said “write it in Rust” and it wrote a plausibly good implementation. I’m not sure I can trust remote interviews any more…

qzw 4y ago
Actually it sounds like recruiting just got a lot easier for you!
abecedarius 4y ago
That's close to answering a question from a bit over a year ago: "in 5 years will there be an AI that's better than 90% of unassisted working programmers at solving new leetcode-type coding interview questions posed in natural language?" https://news.ycombinator.com/item?id=29020401 https://news.ycombinator.com/item?id=29020401 It didn't get a very welcoming reception -- "highly unlikely", and the tenor of other opinions in those comments then tended the same way. But I was being too conservative. (I thought so at the time but it felt sort of outside an Overton window of reasonable technology opinions.)
- imtringued 4y ago
  What? Leetcode is purely about memorization. Of course a model trained on those solutions will regurgitate them. That how humans do it!
  - abecedarius 4y ago
    you missed the word 'new'
naiv 4y ago
Same here. It passed the tests for backend / frontend and created perfect unit tests as well.
xrd 4y ago

Der_Einzige 4y ago

It's been an open secret on blind that cheating on remote interviews is highly, highly common, and has been with language models since late 2019. Best part is that these same folks end up getting promoted because as it turns out once you have passed the stupid coding hazing ritual, most jobs at the FAANG will NOT need you to solve leetcode style problems. I'm honestly excited as heck that something will finally kill off this practice. Don't let the door hit you on the ass on the way out, leetcode.