5 ms·
I recruit senior Java developers. The are so many parallels. They don’t read instructions. Instructions say clearly not to use ai for the written screen. They
by koliber 2y ago
I recruit senior Java developers. The are so many parallels.
They don’t read instructions. Instructions say clearly not to use ai for the written screen.
They copy and paste blindly. The trick is that there are some instructions written in 0-sizes font that people don’t see in the assignment description. The copied-and-pasted version has them but no one rereads the prompt.
Also the AI gives a substandard answer, but that’s besides the point.
- jiggawatts 2y ago> written in 0-sizes font You're... evil. I love it.
- fragmede 2y agoUnfortunately since it's not visible, a unscrupulous motivated candidate will just give an LVM an image of the prompt, let it OCR it, and then pass on its answer. OCRing it also defends against prompt injection with white text on a white background.
- koliber 2y agoA motivated, careful, vigilant candidate will do that and I won't be able to tell. I look for motivated, careful, and vigilant candidates so if they do it and I can't tell, I am OK with it. Most are not motivated, careful, and vigilant. Most are careless, don't pay attention to details, and look for shortcuts.
- FirmwareBurner 2y agoWhat if they use AI but the right way to get the correct answer? What then? Maybe asking candidates something an AI can also answer is not the right way to screen people.
- viraptor 2y agoIf they use it the right way, then you wouldn't be able to tell.
- FirmwareBurner 2y agoThat's my point. He told candidates not to use AI. What's the point of saying that if candidates can use AI without getting caught?
- rizky05 2y ago[dead]
- viraptor 2y agoEasier to communicate both internal and externally why someone was rejected. Lots of acts are commonly understood as wrong, but we still write the laws to make the definitions/rules easy to discuss and reference. Sure, someone may get away with it, but then they proved they can use the system effectively and without causing issues, so... eh..
- koliber 2y agoMost don’t. I know the shape of the answer that ai provides. I added the marker to be able to tell with more confidence. Sometimes I see the marker and the answer is much different and more correct. In those rare cases I bring up the point of trust in the interview if the candidate seems promising.
- FirmwareBurner 2y ago>I know the shape of the answer that ai provides. Curios how you can tell so confidently considering LLMs are constantly changing and candidates can always refactor code to make it seem more human.
- koliber 2y agoIf they refactor, I probably can't tell. Sometimes they refactor a little. I once saw someone refactor it a lot but there was still a tell-tale sign - I ended up inviting the candidate for the interview as they put in a bit more effort rather than blindly pasting in the answer ChatGPT gave them. I can't be 100% but in any given time, ChatGPT seem to produce very similar code for the same prompt. It names the function in one particular way. It adds way too many trivial comments. There are minor variations in the wording and sometimes variables, but in general if I see certain things it's a strong signal that they used AI. I added some additional things (see my original comment) and it correlates perfectly with my observations.
- koliber 2y agoI want people to use ai on the job. To use it effectively you need to be at a certain level so you can correct the ai. That’s why I ask people not to use ai. Additionally, and more importantly, trust is important. If you violate it by ignoring a trivial but prominent ask it sets the wrong foundation for a potential working relationship.
- fragmede 2y agoThe better test I saw was here's the prompt, here's what ChatGPT produced. What's wrong with it, why, and how would you fix it?
- koliber 2y agoI like that!