3 ms·

If I were hiring engineers for a big company, I’d simply ask to see the interviewees’ chat transcripts with ChatGPT or Claude from the past 6 months. I can lear

by EGreg 1y ago

If I were hiring engineers for a big company, I’d simply ask to see the interviewees’ chat transcripts with ChatGPT or Claude from the past 6 months. I can learn so much from that, for how they go about actual modern coding. If I see them arguing with the LLM and nudging it to fix cases, I can see how they’d actually have to code, and the more nuanced their fixes the better. I can spot their attention to detail, how they think thru architecture and software design, the works. If they just take the code given and accept it, that’s a red flag. If word gets out about how we interview, I’d simply ask for even older chats.

danaris 1y ago
...So anyone who coded on their own, you'd just reject out of hand? And (granting, for the moment, your premise that everyone worth hiring used the LLMs) how do you know what they do with the code after they take it from the chat window? My limited experience with the code from LLMs is that there's only so much massaging it's worth giving it through the chat interface, and then you need to just take it and edit it by hand.
- EGreg 1y ago
  I can see their commits presumably?
  - danaris 1y ago
    Only if the software they're working on is open-source. Many, many software engineers work on internal programs that will never see a public repository.
    - EGreg 1y ago
      Well, if they have never released open source, they might not be the right developer for us. We don't have to chase after every type of person, we have specific needs. Having said that, if I was hiring for a big corporation, I'd still get a lot of information from how they interact with the LLMs, and I wouldn't want them to 'game it' a couple weeks in advance so I'd look for older entries. I love how LLMs for the first time give me a real way to assess what people have been doing. And can even use them to vet crazy ideas people send to me. Here is a real example that I did today, see what I mean: https://chatgpt.com/share/68b878b8-8608-800a-8b6a-6bac1af36ec3 https://chatgpt.com/share/68b878b8-8608-800a-8b6a-6bac1af36e...

by EGreg 1y ago

danaris 1y ago

...So anyone who coded on their own, you'd just reject out of hand? And (granting, for the moment, your premise that everyone worth hiring used the LLMs) how do you know what they do with the code after they take it from the chat window? My limited experience with the code from LLMs is that there's only so much massaging it's worth giving it through the chat interface, and then you need to just take it and edit it by hand.

EGreg 1y ago
I can see their commits presumably?
- danaris 1y ago
  Only if the software they're working on is open-source. Many, many software engineers work on internal programs that will never see a public repository.
  - EGreg 1y ago
    Well, if they have never released open source, they might not be the right developer for us. We don't have to chase after every type of person, we have specific needs. Having said that, if I was hiring for a big corporation, I'd still get a lot of information from how they interact with the LLMs, and I wouldn't want them to 'game it' a couple weeks in advance so I'd look for older entries. I love how LLMs for the first time give me a real way to assess what people have been doing. And can even use them to vet crazy ideas people send to me. Here is a real example that I did today, see what I mean: https://chatgpt.com/share/68b878b8-8608-800a-8b6a-6bac1af36ec3 https://chatgpt.com/share/68b878b8-8608-800a-8b6a-6bac1af36e...