3 ms·
Asking dumb questions has become one of my moves in interviews. This is to expose LLM users, as LLMs are extremely unlikely to answer a "how do I do X" question
by remram 1y ago
Asking dumb questions has become one of my moves in interviews. This is to expose LLM users, as LLMs are extremely unlikely to answer a "how do I do X" question with "you shouldn't do X", or "how would you replace Y" with "you should keep using Y".
I guess now I have an additional reason to ask, so that I can screen out the assholes that will say it's a dumb question...
- sandinmyjoints 1y agoWhat interview format are you using that a candidate could be using an LLM during it?
- elijaht 1y agoAnything not in person?
- cluckindan 1y agoThere is software that sends video call transcripts to LLMs and shows you answers in real-time.
- sandinmyjoints 1y agoI was not aware of this. (We haven’t been interviewing recently.) Honest question, is it not possible to see the candidate’s eyes scanning the text? In video meetings I notice this when someone is reading.
- sdedovic 1y agoFun fact, there are multiple pieces of software that will real-time correct your eye tracking. Useful for virtual all hands or talks to fix teleprompter eyes.
- sandinmyjoints 1y agoYikes!
- remram 1y agoYes, I caught one candidate doing exactly that. There were other obvious signs as well. Of course I don't know how many I didn't catch.
- danielheath 1y agoVideo call?
- Freedom2 1y agoLast time I said "you shouldn't do X" in an in person interview, I failed and the feedback was "candidate was not cooperative with the interviewer". So even if the candidate is not using an LLM, the interview process is so messed up now that people willingly go along otherwise they may be rejected for, you know, actually stating what their experience was like. Funnily enough, that was a YC funded company that I was interviewing for, and I expected a lot better.
- ajmurmann 1y agoThat's very disappointing. As an interviewer the ideal situation I want in an interview is one where we are discussing tradeoffs. It allows the interviewee to show a deep understanding of the solutions we are discussing and, depending on the problem, even share insights around business impact. I wonder if the pushback is more of a common reaction from more junior interviewers or once that aren't very empowered and just need to go down a script and worry about getting in trouble if they deviate.
- mingus88 1y agoYeah it sounds like maybe OP was too direct. “Uncooperative” is strong language for feedback I had a systems scenario recently where I was expected to troubleshoot a MySQL master that was trying to backup via rsync to a hung NFS mount. I successfully resolved the problem and only then detailed how messed up their design was to get into such a situation in 2025.
- stevesearer 1y agoNot sure how your delivery was, but maybe the best way is, “If I were doing it that way, here’s how I’d go about it. However, I might consider this other way and here’s why.”
- Freedom2 1y agoIn all honesty, my words were something along the lines of "I don't think that approach is a good idea. It's because of X reason." Another viewpoint I've seen when sharing this story is that the interviewer wants me to start with the "bad" implementation first, then we move on to the "proper" one, but what's the point if I actually have pointed out the flaws beforehand?
- roryirvine 1y agoThis sort of question works well for assessing senior / lead skills. For example, when asked how to go about replacing PostgreSQL with MongoDB, it'd be okay for someone fairly junior to either give a straightforward answer or to turn their nose up and say "eww, no" - but you'd expect someone with more seniority to instead flag that it was an unusual scenario and ask for more detail. Prepare a vaguely-plausible reason (eg. "we're storing JSON documents and the team are more comfortable with Mongo's tooling"), and this can form the basis for a productive discussion which you can drive in whichever social or technical direction you feel is most appropriate.