3 ms·

Last year I was reviewing about a dozen answers to the questions we were sending candidates to complete at home. In all cases those a*holes used ChatGPT to writ

Last year I was reviewing about a dozen answers to the questions we were sending candidates to complete at home. In all cases those a*holes used ChatGPT to write answers and what did ChatGPT use? Original Python docs recombined with blog posts. I was able to verify this by copying passages from their answers and pasting them into Google. When I showed this to the hiring manager they had a serious conversation with the agency. It would have been hilarious if I did not know that some of the candidate were real people with good experience but too lazy to write simple code.

esafak 2y ago
You can't assign homework any more. I was never into them anyway. One clumsy candidate accidentally revealed his cards by pasting the prompt he'd used in our chat window, like that paper whose introduction began "Certainly, here is a possible introduction to your topic." https://www.reddit.com/r/China/comments/1beaqya/obvious_chatgpt_prompt_reply_in_published_paper/ https://www.reddit.com/r/China/comments/1beaqya/obvious_chat... Another was unable to explain why his code was structured the way it was. I asked a multipart question that required refactoring, knowing that these tools are not good at that, or enforcing a consistent style. His initial code was a self-contained class and the refactor introduced a function outside of the class that also should have been a method. I asked him why he did that and, of course, he froze.
- lithos 2y ago
  AI code also alternates between technical levels IE: you'll see a part that looks like it was trained on beginner tutorials followed by another part that's looks trained on ripped open source projects. Sometimes just really silly things like 3D vector math written in a form that would easily support higher dimensions for whatever reason.
knallfrosch 2y ago
Did you tell them that AI was not allowed?
- surfingdino 2y ago
  We did not, but using AI to generate or reformat answers is a big red flag to me, because it shows that a) you do not have the experience you claim to have, and b) you do not have written communication skills or knowledge of English required to do your job.
rnewme 2y ago
People you're calling assholes - did they get paid for the take home task? We're they instructed not to use llms? Have you confirmed they used them and didn't Google the same things you googled? Was the homework not correct?
- surfingdino 2y ago
  They were not paid, but we spared them the stress of a timed online coding exercise which a lot of devs hate. We told them we expect short snippets of code to illustrate the answers to six questions which progressed from the obvious (if you code in the language for a few years) to more of "there are many was to do it, here is mine" to judge their ability to turn requirements into code. You could do the whole thing in half an hour. It was fair.
  - Clubber 2y ago
    We're really struggling to find good people. We do it the old fashioned way, just a one hour 4 dev interview. We don't ask really tough questions or anything. An example is, "can you describe normalization and what it's for?" I think the market is just supersaturated with bads right now and all the bads are the ones applying. The good developers we have we got from knowing them in previous jobs, etc. You might wanna hit up your current team to find better devs. You should probably pay your team a finder's fee if you hire from that. Also, if you are paying what you paid before covid, it's probably too low today. Times are tough for building good teams right now. You need to adapt. The same ole "call a recruiter and get 1 good guy out of 10," isn't a thing anymore. You might wanna pay 10-20% over market and see if that works. To justify that, just calculate what you are paying to not find good people now.
    - JohnFen 2y ago
      > I think the market is just supersaturated with bads right now and all the bads are the ones applying. This is what it looks like where I work, too. There's been a huge decline in the quality of applicants over the past few years. I think the percentage of applicants that could be called "competent" has fallen to below 20%.

esafak 2y ago

You can't assign homework any more. I was never into them anyway. One clumsy candidate accidentally revealed his cards by pasting the prompt he'd used in our chat window, like that paper whose introduction began "Certainly, here is a possible introduction to your topic." https://www.reddit.com/r/China/comments/1beaqya/obvious_chatgpt_prompt_reply_in_published_paper/ https://www.reddit.com/r/China/comments/1beaqya/obvious_chat... Another was unable to explain why his code was structured the way it was. I asked a multipart question that required refactoring, knowing that these tools are not good at that, or enforcing a consistent style. His initial code was a self-contained class and the refactor introduced a function outside of the class that also should have been a method. I asked him why he did that and, of course, he froze.

lithos 2y ago
AI code also alternates between technical levels IE: you'll see a part that looks like it was trained on beginner tutorials followed by another part that's looks trained on ripped open source projects. Sometimes just really silly things like 3D vector math written in a form that would easily support higher dimensions for whatever reason.

knallfrosch 2y ago

Did you tell them that AI was not allowed?

surfingdino 2y ago
We did not, but using AI to generate or reformat answers is a big red flag to me, because it shows that a) you do not have the experience you claim to have, and b) you do not have written communication skills or knowledge of English required to do your job.

rnewme 2y ago

People you're calling assholes - did they get paid for the take home task? We're they instructed not to use llms? Have you confirmed they used them and didn't Google the same things you googled? Was the homework not correct?

surfingdino 2y ago
They were not paid, but we spared them the stress of a timed online coding exercise which a lot of devs hate. We told them we expect short snippets of code to illustrate the answers to six questions which progressed from the obvious (if you code in the language for a few years) to more of "there are many was to do it, here is mine" to judge their ability to turn requirements into code. You could do the whole thing in half an hour. It was fair.
- Clubber 2y ago
  We're really struggling to find good people. We do it the old fashioned way, just a one hour 4 dev interview. We don't ask really tough questions or anything. An example is, "can you describe normalization and what it's for?" I think the market is just supersaturated with bads right now and all the bads are the ones applying. The good developers we have we got from knowing them in previous jobs, etc. You might wanna hit up your current team to find better devs. You should probably pay your team a finder's fee if you hire from that. Also, if you are paying what you paid before covid, it's probably too low today. Times are tough for building good teams right now. You need to adapt. The same ole "call a recruiter and get 1 good guy out of 10," isn't a thing anymore. You might wanna pay 10-20% over market and see if that works. To justify that, just calculate what you are paying to not find good people now.
  - JohnFen 2y ago
    > I think the market is just supersaturated with bads right now and all the bads are the ones applying. This is what it looks like where I work, too. There's been a huge decline in the quality of applicants over the past few years. I think the percentage of applicants that could be called "competent" has fallen to below 20%.