3 ms·

We have been running LLM and coding agents for a while now and my overall observation is that it is a powertool or a crane, it is not a decision making tool. N

We have been running LLM and coding agents for a while now and my overall observation is that it is a powertool or a crane, it is not a decision making tool. Now in my org, people who have great understanding of concepts, deeper engineering understand have exponential productivity. People who dont or new in the workforce, juniors, are generating hell-ish code without understand as long as it runs they think the job is done. And this is where the problem is. The llm creates an intellectual gap within the org and it just widens it as more and more it gets used. You might end up not trusting stuff within the org if code is generated by later.

abustamam 5mo ago
This is my experience. I'll use LLMs as a sounding board for architectural decisions and to bring discussion points up to the team, and we talk through assumptions and pros and cons. And then once we have the architecture in place, LLMs are pretty good at implementation.
ghosty141 5mo ago
Exactly my (and my coworkers) experience. AI generally amplifies the skillset, both in the good and the bad. One fantastic usecase for me just recently was writing up a concept for an authentication daemon. With codex this is like a conversation where I pick from the suggestions, cross reference them with normal web-search and decide on a final draft which I then discuss with colleagues. This "conversational" planning with integrated web-search (aka plan mode) is insanely useful. Also reviewing already written code with AI is purely beneficial in my opinion. In my opinion the main caveat of AI is, you eventually have to be smarter than then tool. So for example if Codex suggests I should use tech-stack X then I must research and fully understand why this is actually good and still have to compare to other solutions. I think this is where the problem lies, some people skip this step which leads to so so many problems, and that's fatal. You MUST be smarter than the AI after your conversation and fully understand and be able to critique what it said.
- silentkat 5mo ago
  The power of AI is it rewards due diligence. The weakness of AI is that it is really easy to fall into lazy habits. Something about having to talk to a machine like it's a human makes me fall for treating it like a human. I want to treat it as a probability engine that collapses to an answer based on input, but that input explicitly needs to be one that has it collapse to something a reasonably knowledgeable person would respond with, which more-or-less means talking to it like it is that kind of person. I feel like it activates the social part of my brain and then I stop working with it properly. I'm still building the habit, though, only recently started taking the LLMs seriously as a tool.
cmrdporcupine 5mo ago
I agree with this assessment but even among us seniors with accumulated knowledge it has the dangerous potential of getting out from under your feet and produce large amounts of code that you don't have full comprehension of. I can generally make it produce excellent well-tested code. Far better than I could do in the same time on my own. But it's a challenge to keep on top of knowledge about everything it made.

abustamam 5mo ago

This is my experience. I'll use LLMs as a sounding board for architectural decisions and to bring discussion points up to the team, and we talk through assumptions and pros and cons. And then once we have the architecture in place, LLMs are pretty good at implementation.

ghosty141 5mo ago

Exactly my (and my coworkers) experience. AI generally amplifies the skillset, both in the good and the bad. One fantastic usecase for me just recently was writing up a concept for an authentication daemon. With codex this is like a conversation where I pick from the suggestions, cross reference them with normal web-search and decide on a final draft which I then discuss with colleagues. This "conversational" planning with integrated web-search (aka plan mode) is insanely useful. Also reviewing already written code with AI is purely beneficial in my opinion. In my opinion the main caveat of AI is, you eventually have to be smarter than then tool. So for example if Codex suggests I should use tech-stack X then I must research and fully understand why this is actually good and still have to compare to other solutions. I think this is where the problem lies, some people skip this step which leads to so so many problems, and that's fatal. You MUST be smarter than the AI after your conversation and fully understand and be able to critique what it said.

silentkat 5mo ago
The power of AI is it rewards due diligence. The weakness of AI is that it is really easy to fall into lazy habits. Something about having to talk to a machine like it's a human makes me fall for treating it like a human. I want to treat it as a probability engine that collapses to an answer based on input, but that input explicitly needs to be one that has it collapse to something a reasonably knowledgeable person would respond with, which more-or-less means talking to it like it is that kind of person. I feel like it activates the social part of my brain and then I stop working with it properly. I'm still building the habit, though, only recently started taking the LLMs seriously as a tool.

cmrdporcupine 5mo ago

I agree with this assessment but even among us seniors with accumulated knowledge it has the dangerous potential of getting out from under your feet and produce large amounts of code that you don't have full comprehension of. I can generally make it produce excellent well-tested code. Far better than I could do in the same time on my own. But it's a challenge to keep on top of knowledge about everything it made.