4 ms·

It's amazing this is still an opinion in 2025. I now ask devs how they use AI as part of their workflows when I interview. It's a standard skill I expect my g

by swalsh 2y ago

It's amazing this is still an opinion in 2025. I now ask devs how they use AI as part of their workflows when I interview. It's a standard skill I expect my guys to have.

BeetleB 2y ago
Just curious, but what AI related skills do you expect them to have?
- eschaton 2y ago
  The ability to recognize and join a hype train, I presume. It’s one way to appear proactively leading-edge to marginally-informed product managers, marketers, execs and press.
  - adwn 2y ago
    That's an extremely uncharitable presumption. Although I don't agree that routine usage of AIs should be a precondition for regular software engineering jobs, there are good reasons for using LLMs besides "joining a hype train".
    - eschaton 2y ago
      Nah.
- swalsh 2y ago
  I ask what their current workflow is, how they check and verify things, what their approach to prompting is etc. I'm looking to see that they've developed basic skills, have a reasonable mental model of what models can do well, what they currently can't do, and have an approach to be productive using the tools.
dartos 2y ago
I feel bad for your team. Let people work how they want. I wouldn’t not hire someone on the basis of them not using a language server. The creator of the Odin language famously doesn’t use one. He’s says that he, specifically, is faster without one.
- theptip 2y ago
  No, it’s reasonable. If your team uses Git then it’s a valid question to establish if someone has only worked with Perforce. They didn’t say how heavily they weight the question. (All that said I expect that, soon, experience with the appropriate LLM tooling will be as important as having experience with the language your system is implemented in.)
  - dartos 2y ago
    Right, but using git is a team wide thing. I can’t use perforce while my company is on git. But if I do or do not use an LLM to assist me while coding, my team is unaffected. If someone liked jetbrains, but your team used neovim, would you force them to use neovim?
    - Kwpolska 2y ago
      Editors may also be a team decision in some places. Some teams are using features unique to one IDE, for example.
      - chikere232 2y ago
        it can be a team decision, but it's a bad one
      - dartos 2y ago
        Then that tooling is required, like visual studio is a common one I know about in windows land. Though nobody should care if I edited my text files with neovim as long as I still used the same toolchain as everyone else.
  - 2y ago
sramam 2y ago
I concur that asking devs how they use AI is a great idea. Recently, I shared a code base with a junior dev and she was surprised with the speed and sophistication of the code. The LLM did 80+% of the "coding". What was telling was as she was grokking the code (for helping the ~20%), she was surprised at the quality of the code - her use of the LLM did not yield code of similar quality. I find that the more domain awareness one brings to the table, the better the output is. Basically the clearer one's vision of the end-state, the better the output. One other positive side-effect of using "LLMs as a junior-dev" for me has been that my ambitions are greater. I want it all - better code, more sophisticated capabilities even for relatively not-important projects, documentation, tests, debug-ability. And once the basic structure is in place, many a time it is trivial to get the rest. It's never 100%, but even with 80+%, I am faster than ever before, deliver better quality code, and can switch domains multiple times a week and never feel drained. Sharing best AI hacks within a team will have the same effect as code-reviews do in ensuring consistency. Perhaps an "LLM chat review", especially when something particularly novel was accomplished!
layer8 2y ago
Using cloud-based AI is a no-go where I work, for IP and contractual reasons. And on-premises AI is not as capable and more difficult to integrate.
- simonw 2y ago
  Have you tried the latest open weight models? They're SO MUCH better today than they were even six months ago. If I was in an environment that didn't allow hosted API models I'd absolutely be looking into the various Llama 3 models or Qwen2.5-Coder-32B.
  - 3eb7988a1663 2y ago
    Legal does not even want us running offline models for reasons. I assume that comes down to not knowing what offline-only means, but such is life.
    - simonw 2y ago
      Maybe they're concerned that code written with AI assistance can't be copyrighted? I've seen that idea floated in a few places.
  - layer8 2y ago
    What do you use so that you can throw in a set of documents and/or a nontrivial code base into an LLM workspace and ask questions about it etc.? What the cloud-based services provide goes way beyond a simple chat interface or mere code completion (as you know, of course).
    - simonw 2y ago
      I use my https://github.com/simonw/files-to-prompt https://github.com/simonw/files-to-prompt tool like this: files-to-prompt . -e py -e md -c | pbcopy Now I have all the Python and Markdown files from the current project on my clipboard, in Claude's recommended XML-like format (which I find works well with other models too). Then I paste that into the Claude web interface or Google's AI Studio if it's too long for Claude and ask questions there. Sometimes I'll pipe it straight into my own LLM CLI tool and ask questions that way: files-to-prompt . -e py -e md -c | \ llm -m gemini-2.0-flash-exp 'which files handle JWT verification?' I can later start a chat session on top of the accumulated context like this: llm chat -c (The -c means "continue most recent conversation in the chat").

by swalsh 2y ago

It's amazing this is still an opinion in 2025. I now ask devs how they use AI as part of their workflows when I interview. It's a standard skill I expect my guys to have.

BeetleB 2y ago

Just curious, but what AI related skills do you expect them to have?

eschaton 2y ago
The ability to recognize and join a hype train, I presume. It’s one way to appear proactively leading-edge to marginally-informed product managers, marketers, execs and press.
- adwn 2y ago
  That's an extremely uncharitable presumption. Although I don't agree that routine usage of AIs should be a precondition for regular software engineering jobs, there are good reasons for using LLMs besides "joining a hype train".
  - eschaton 2y ago
    Nah.
swalsh 2y ago
I ask what their current workflow is, how they check and verify things, what their approach to prompting is etc. I'm looking to see that they've developed basic skills, have a reasonable mental model of what models can do well, what they currently can't do, and have an approach to be productive using the tools.

dartos 2y ago

I feel bad for your team. Let people work how they want. I wouldn’t not hire someone on the basis of them not using a language server. The creator of the Odin language famously doesn’t use one. He’s says that he, specifically, is faster without one.

theptip 2y ago
No, it’s reasonable. If your team uses Git then it’s a valid question to establish if someone has only worked with Perforce. They didn’t say how heavily they weight the question. (All that said I expect that, soon, experience with the appropriate LLM tooling will be as important as having experience with the language your system is implemented in.)
- dartos 2y ago
  Right, but using git is a team wide thing. I can’t use perforce while my company is on git. But if I do or do not use an LLM to assist me while coding, my team is unaffected. If someone liked jetbrains, but your team used neovim, would you force them to use neovim?
  - Kwpolska 2y ago
    Editors may also be a team decision in some places. Some teams are using features unique to one IDE, for example.
    - chikere232 2y ago
      it can be a team decision, but it's a bad one
    - dartos 2y ago
      Then that tooling is required, like visual studio is a common one I know about in windows land. Though nobody should care if I edited my text files with neovim as long as I still used the same toolchain as everyone else.
- 2y ago

sramam 2y ago

I concur that asking devs how they use AI is a great idea. Recently, I shared a code base with a junior dev and she was surprised with the speed and sophistication of the code. The LLM did 80+% of the "coding". What was telling was as she was grokking the code (for helping the ~20%), she was surprised at the quality of the code - her use of the LLM did not yield code of similar quality. I find that the more domain awareness one brings to the table, the better the output is. Basically the clearer one's vision of the end-state, the better the output. One other positive side-effect of using "LLMs as a junior-dev" for me has been that my ambitions are greater. I want it all - better code, more sophisticated capabilities even for relatively not-important projects, documentation, tests, debug-ability. And once the basic structure is in place, many a time it is trivial to get the rest. It's never 100%, but even with 80+%, I am faster than ever before, deliver better quality code, and can switch domains multiple times a week and never feel drained. Sharing best AI hacks within a team will have the same effect as code-reviews do in ensuring consistency. Perhaps an "LLM chat review", especially when something particularly novel was accomplished!

layer8 2y ago

Using cloud-based AI is a no-go where I work, for IP and contractual reasons. And on-premises AI is not as capable and more difficult to integrate.

simonw 2y ago
Have you tried the latest open weight models? They're SO MUCH better today than they were even six months ago. If I was in an environment that didn't allow hosted API models I'd absolutely be looking into the various Llama 3 models or Qwen2.5-Coder-32B.
- 3eb7988a1663 2y ago
  Legal does not even want us running offline models for reasons. I assume that comes down to not knowing what offline-only means, but such is life.
  - simonw 2y ago
    Maybe they're concerned that code written with AI assistance can't be copyrighted? I've seen that idea floated in a few places.
- layer8 2y ago
  What do you use so that you can throw in a set of documents and/or a nontrivial code base into an LLM workspace and ask questions about it etc.? What the cloud-based services provide goes way beyond a simple chat interface or mere code completion (as you know, of course).
  - simonw 2y ago
    I use my https://github.com/simonw/files-to-prompt https://github.com/simonw/files-to-prompt tool like this: files-to-prompt . -e py -e md -c | pbcopy Now I have all the Python and Markdown files from the current project on my clipboard, in Claude's recommended XML-like format (which I find works well with other models too). Then I paste that into the Claude web interface or Google's AI Studio if it's too long for Claude and ask questions there. Sometimes I'll pipe it straight into my own LLM CLI tool and ask questions that way: files-to-prompt . -e py -e md -c | \ llm -m gemini-2.0-flash-exp 'which files handle JWT verification?' I can later start a chat session on top of the accumulated context like this: llm chat -c (The -c means "continue most recent conversation in the chat").