3 ms·

> I want the prompts you used to build it. The better your prompts, the better your grade. You should have not said that. They will use a model to create the

by belter 1y ago

> I want the prompts you used to build it. The better your prompts, the better your grade. You should have not said that. They will use a model to create the prompts... :-) You should have said: I will break your code and you have to fix it in front of me.

joshstrange 1y ago
> I will break your code and you have to fix it in front of me. For the coding aspect of our interview test, I tell people they can use whatever they want to write it (LLM tools included) and they do all of the work on their own machine without us watching the screen in real time (to make it less stressful). After they’re finished, I ask them to walk me through the code, to make one addition, and fix a bug that almost everyone makes. This helps demonstrate their ability to explain code (even if they were assisted in writing it) and to modify existing code. This exercise is shockingly effective. None of this is done as a “gotcha“. The change that I asked them to make is incredibly minor, and in most cases can be done completely additively to the existing code. As for the common bug, it is due to people “pattern matching” and misunderstanding part of the instructions, but also very easy to fix once you understand the mistake. I’ve been very pleased with using this as a filter. In some cases, the results were surprising and, in others, they confirmed what I already thought/suspected.
- pyman 1y ago
  Sounds like a good process, but I'm curious: are you looking for someone who understands code, or someone who knows how to solve problems and happens to understand code?
  - joshstrange 1y ago
    Someone who knows how to solve problems and can also code, but I feel I can get a good feel of that from the other tests we give. We also have a problem solving test with a real example we dealt with that is simple enough to understand and is another good filter. The first coding tests proves they can at least string together a bit of code, explain it, and make minor changes to it. The problem solving test proves they can think critically and understand a new problem and offer solutions to it. Lastly, we have a debugging test where they have to fix existing code with a handful of bugs. Between these and the conversations we have have with applicants we have (so far) a very good success rate in hiring.
pyman 1y ago
I think it depends on the age. The younger the student, the more important it is to learn how to solve problems than to code. Those who choose to become software developers will eventually learn how to read code. For me, what matters early on is learning how to break down a problem, understand what matters, and think clearly. The rest is just memorising text.

by belter 1y ago

joshstrange 1y ago

> I will break your code and you have to fix it in front of me. For the coding aspect of our interview test, I tell people they can use whatever they want to write it (LLM tools included) and they do all of the work on their own machine without us watching the screen in real time (to make it less stressful). After they’re finished, I ask them to walk me through the code, to make one addition, and fix a bug that almost everyone makes. This helps demonstrate their ability to explain code (even if they were assisted in writing it) and to modify existing code. This exercise is shockingly effective. None of this is done as a “gotcha“. The change that I asked them to make is incredibly minor, and in most cases can be done completely additively to the existing code. As for the common bug, it is due to people “pattern matching” and misunderstanding part of the instructions, but also very easy to fix once you understand the mistake. I’ve been very pleased with using this as a filter. In some cases, the results were surprising and, in others, they confirmed what I already thought/suspected.

pyman 1y ago
Sounds like a good process, but I'm curious: are you looking for someone who understands code, or someone who knows how to solve problems and happens to understand code?
- joshstrange 1y ago
  Someone who knows how to solve problems and can also code, but I feel I can get a good feel of that from the other tests we give. We also have a problem solving test with a real example we dealt with that is simple enough to understand and is another good filter. The first coding tests proves they can at least string together a bit of code, explain it, and make minor changes to it. The problem solving test proves they can think critically and understand a new problem and offer solutions to it. Lastly, we have a debugging test where they have to fix existing code with a handful of bugs. Between these and the conversations we have have with applicants we have (so far) a very good success rate in hiring.

pyman 1y ago

I think it depends on the age. The younger the student, the more important it is to learn how to solve problems than to code. Those who choose to become software developers will eventually learn how to read code. For me, what matters early on is learning how to break down a problem, understand what matters, and think clearly. The rest is just memorising text.