4 ms·

I see worrying trends in my office. Developers (often juniors) use LLM code without taking time to verify it. This leads to bugs and they can't fix it because

I see worrying trends in my office. Developers (often juniors) use LLM code without taking time to verify it. This leads to bugs and they can't fix it because they don't understand the code. Some senior developers also trust the tool to generate a function, and don't take the time to review it and catch the edge cases that the tool missed. They rely on ChatGPT to answer their questions instead of taking time to read the documentation or a simple web search to see discussions on stack overflow or blogs about the subject. This may give results in the short term, but they don't actually learn to solve problems themselves. I am afraid that this will have huge negative effects on their career if the tools improve significantly. Learning how to solve problems is an important skill. They also lose access to the deeper knowledge that enable you to see connections, complexities and flows that the current generation of tools are unable to do. By reading the documentation, blogs or discussions you are often exposed to a wider view of the subject than the laser focused answer of ChatGPT There will be less room for "vibe coders" in the future, as these tools increasingly solve the simple things without requiring as much management. Until we reach AGI (I doubt it will happen within the next 10 years) the tools will require experienced developers to guide them for the more complex issues. Older experienced developers, and younger developers who have learned how to solve problems and have deep knowledge, will be in demand.

sramam 1y ago
Aren't the insufficiencies of the LLMs a temporary condition? And as with any automation, there will be a select few who will understand it's inner workings, and a vast majority that will enjoy/suffer the benefits.
jagged-chisel 1y ago
> They rely on ChatGPT to answer their questions instead of taking time to read the documentation or a simple web search. Documentation is not written with answers in mind. Every little project wants me to be an expert in their solution. They want to share with me the theory behind their decisions. I need an answer now. Web search no longer provides useful information within the first few results. Instead, I get content farms who are worse than recipe pages - explaining why someone would want this information, but never providing it. A junior isn’t going to learn from information that starts from the beginning (“if you want to make an apple pie from scratch, you must first invent the universe.”) 99.999% of them need a solution they can tweak as needed so they can begin to understand the thing. LLMs are good at processing and restructuring information so I can ask for things the way I prefer to receive them. Ultimately, the problem is actually all about verification.
- guappa 1y ago
  Ah yes, LLM is very good at giving me information from documentation that was out of date 15 years ago instead of using the documentation from 2025.
  - TingPing 1y ago
    Mostly made up information in my experience.
  - calvinmorrison 1y ago
    it's been enormously useful for my Qt3 work though, it really understands it well.
  - pc86 1y ago
    Most LLMs, especially the paid tiers, will fetch updated information. This was a valid complaint perhaps 8-12 months ago.
    - LtWorf 1y ago
      You mean they will DOS the servers of the open source projects? That's even worse!
  - davidguetta 1y ago
    Funny that that comment is itself out of date for approx. 15 month ago ><
- ori_b 1y ago
  > Documentation is not written with answers in mind. Every little project wants me to be an expert in their solution. They want to share with me the theory behind their decisions. I need an answer now. I have an answer now, because I read the documentation last week.
  - fn-mote 1y ago
    This is kind of dismissive. As a real example, I needed to change my editor config last month. I do this about once every 5 years. I really didn’t want to become an expert in the config system again, so I tried LLM. Sad to report, it told me where to look but all of the exact details were wrong. Maybe someday soon, though.
asim 1y ago
Same could be said for every language abstraction or systems layer change. When we stopped programming kernel modules and actually found a workable interface it opened the door to so many more developers. I'm sure at the time there was skepticism because people didn't understand the internals of the kernel. That's not the point. The point is to raise the level of abstraction to open the door, increase productivity and focus on new problems. When you see 30-50 years of change you realise this was inevitable and in every generation there's new engineers entering with limited understanding of the layers beneath. Even the code produced. Do I understand the lexers and the compilers that turn my code in to machine code or instruction sets? Heck no. Doesn't mean I shouldn't use the tools available to me now.
- ajsisbdjxbx 1y ago
  No, but you can understand them if given time. And you can rely on them to be some degree of reliable approaching 100% (and when they fail it will likely be in a consistent way you can understand with sufficient time, and likely fix). LLMs don’t have these properties. Randomness makes for a poor abstraction layer. We invent tools because humans suffer from this issue too.
- oblio 1y ago
  > it opened the door to so many more developers. [...] That's not the point. The point is to raise the level of abstraction to open the door, increase productivity and focus on new problems. There are diminishing returns. At some point, quoting Cool Hand Luke, some men you just can't (r|)teach.
agumonkey 1y ago
So the scope of answers are single function or single class ? I have people nearby that are attempting generating whole projects, I really wonder how they will ensure anything about it beside the happy paths. Or maybe they plan to have an army of agents fuzzing and creating hotfixes 24/7 ..
- pc86 1y ago
  > Or maybe they plan to have an army of agents fuzzing and creating hotfixes 24/7 There are absolutely people who plan to do exactly this. Use AI to create a half-baked, AI-led solution, and continue to use AI to tweak it. For people with sufficient capital it might actually work out halfway decent. I've had success with greenfield AI generation but only in a very specific manner: 1. Talk with the LLM about what you're building and have it generate a detailed technical specification. Iterate on this until you have a good, human-readable explanation of the entire application or feature. 2. Start a completely new chat/context. If you're using something like Gemini, turn temperature down and enable external search. 3. Have instructions¹ guiding the LLM, this might be the most important step, even moreso than #1. 4. Create the base/blank project as its own step. Zero features or config. 5. Copy features one at a time from the spec to the chat context OR have them as separate documents and say things like "we're creating Feature 3A.1" or whatever. 6. Iterate on each feature until you're happy then repeat. ¹ https://www.totaltypescript.com/cursor-rules-for-better-ai-development https://www.totaltypescript.com/cursor-rules-for-better-ai-d...
jamesrr39 1y ago
> Developers (often juniors) use LLM code without taking time to verify it. This leads to bugs and they can't fix it because they don't understand the code Well... is this something new? Previously the trend was to copy and paste Stackoverflow answers, without understanding what it did. Perhaps with LLM code it's an incremental change but the concept is fairly familiar.
dmcgill50 1y ago
This is just one more turtle up. In college, I took a class where they taught us how to code in Assembler. I haven't looked at Assembler until this morning and here is a summary of my 5 minutes of work. Here's an overview of what we've done: 1. *Created Assembly Code for Apple Silicon*: We wrote ARM64 assembly code specifically for your Apple M1 Max processor running macOS, rather than x86 assembly which wouldn't work on your architecture. 2. *Explained the Compilation Process*: We covered how to compile and link the code using the `as` assembler and `ld` linker with the proper flags for macOS on ARM64. 3. *Addressed Development Environment*: We confirmed that you don't need to install a separate assembler since it comes with Apple's Command Line Tools, and provided instructions on how to verify or install these tools. 4. *Optimized the Code*: We refined the code with better alignment for potential performance improvements, though noted that for a "Hello World" program, system call overhead is the main performance factor. 5. *Used macOS-Specific Syscalls*: The assembly code uses the appropriate syscall numbers and conventions specific to macOS on ARM64 architecture (syscalls 4 for write and 1 for exit). This gives you a basic introduction to writing assembly directly for Apple Silicon, which is quite different from traditional x86 assembly programming.

sramam 1y ago

Aren't the insufficiencies of the LLMs a temporary condition? And as with any automation, there will be a select few who will understand it's inner workings, and a vast majority that will enjoy/suffer the benefits.

jagged-chisel 1y ago

> They rely on ChatGPT to answer their questions instead of taking time to read the documentation or a simple web search. Documentation is not written with answers in mind. Every little project wants me to be an expert in their solution. They want to share with me the theory behind their decisions. I need an answer now. Web search no longer provides useful information within the first few results. Instead, I get content farms who are worse than recipe pages - explaining why someone would want this information, but never providing it. A junior isn’t going to learn from information that starts from the beginning (“if you want to make an apple pie from scratch, you must first invent the universe.”) 99.999% of them need a solution they can tweak as needed so they can begin to understand the thing. LLMs are good at processing and restructuring information so I can ask for things the way I prefer to receive them. Ultimately, the problem is actually all about verification.

guappa 1y ago
Ah yes, LLM is very good at giving me information from documentation that was out of date 15 years ago instead of using the documentation from 2025.
- TingPing 1y ago
  Mostly made up information in my experience.
- calvinmorrison 1y ago
  it's been enormously useful for my Qt3 work though, it really understands it well.
- pc86 1y ago
  Most LLMs, especially the paid tiers, will fetch updated information. This was a valid complaint perhaps 8-12 months ago.
  - LtWorf 1y ago
    You mean they will DOS the servers of the open source projects? That's even worse!
- davidguetta 1y ago
  Funny that that comment is itself out of date for approx. 15 month ago ><
ori_b 1y ago
> Documentation is not written with answers in mind. Every little project wants me to be an expert in their solution. They want to share with me the theory behind their decisions. I need an answer now. I have an answer now, because I read the documentation last week.
- fn-mote 1y ago
  This is kind of dismissive. As a real example, I needed to change my editor config last month. I do this about once every 5 years. I really didn’t want to become an expert in the config system again, so I tried LLM. Sad to report, it told me where to look but all of the exact details were wrong. Maybe someday soon, though.

asim 1y ago

Same could be said for every language abstraction or systems layer change. When we stopped programming kernel modules and actually found a workable interface it opened the door to so many more developers. I'm sure at the time there was skepticism because people didn't understand the internals of the kernel. That's not the point. The point is to raise the level of abstraction to open the door, increase productivity and focus on new problems. When you see 30-50 years of change you realise this was inevitable and in every generation there's new engineers entering with limited understanding of the layers beneath. Even the code produced. Do I understand the lexers and the compilers that turn my code in to machine code or instruction sets? Heck no. Doesn't mean I shouldn't use the tools available to me now.

ajsisbdjxbx 1y ago
No, but you can understand them if given time. And you can rely on them to be some degree of reliable approaching 100% (and when they fail it will likely be in a consistent way you can understand with sufficient time, and likely fix). LLMs don’t have these properties. Randomness makes for a poor abstraction layer. We invent tools because humans suffer from this issue too.
oblio 1y ago
> it opened the door to so many more developers. [...] That's not the point. The point is to raise the level of abstraction to open the door, increase productivity and focus on new problems. There are diminishing returns. At some point, quoting Cool Hand Luke, some men you just can't (r|)teach.

agumonkey 1y ago

So the scope of answers are single function or single class ? I have people nearby that are attempting generating whole projects, I really wonder how they will ensure anything about it beside the happy paths. Or maybe they plan to have an army of agents fuzzing and creating hotfixes 24/7 ..

pc86 1y ago
> Or maybe they plan to have an army of agents fuzzing and creating hotfixes 24/7 There are absolutely people who plan to do exactly this. Use AI to create a half-baked, AI-led solution, and continue to use AI to tweak it. For people with sufficient capital it might actually work out halfway decent. I've had success with greenfield AI generation but only in a very specific manner: 1. Talk with the LLM about what you're building and have it generate a detailed technical specification. Iterate on this until you have a good, human-readable explanation of the entire application or feature. 2. Start a completely new chat/context. If you're using something like Gemini, turn temperature down and enable external search. 3. Have instructions¹ guiding the LLM, this might be the most important step, even moreso than #1. 4. Create the base/blank project as its own step. Zero features or config. 5. Copy features one at a time from the spec to the chat context OR have them as separate documents and say things like "we're creating Feature 3A.1" or whatever. 6. Iterate on each feature until you're happy then repeat. ¹ https://www.totaltypescript.com/cursor-rules-for-better-ai-development https://www.totaltypescript.com/cursor-rules-for-better-ai-d...

jamesrr39 1y ago

> Developers (often juniors) use LLM code without taking time to verify it. This leads to bugs and they can't fix it because they don't understand the code Well... is this something new? Previously the trend was to copy and paste Stackoverflow answers, without understanding what it did. Perhaps with LLM code it's an incremental change but the concept is fairly familiar.

dmcgill50 1y ago

This is just one more turtle up. In college, I took a class where they taught us how to code in Assembler. I haven't looked at Assembler until this morning and here is a summary of my 5 minutes of work. Here's an overview of what we've done: 1. *Created Assembly Code for Apple Silicon*: We wrote ARM64 assembly code specifically for your Apple M1 Max processor running macOS, rather than x86 assembly which wouldn't work on your architecture. 2. *Explained the Compilation Process*: We covered how to compile and link the code using the `as` assembler and `ld` linker with the proper flags for macOS on ARM64. 3. *Addressed Development Environment*: We confirmed that you don't need to install a separate assembler since it comes with Apple's Command Line Tools, and provided instructions on how to verify or install these tools. 4. *Optimized the Code*: We refined the code with better alignment for potential performance improvements, though noted that for a "Hello World" program, system call overhead is the main performance factor. 5. *Used macOS-Specific Syscalls*: The assembly code uses the appropriate syscall numbers and conventions specific to macOS on ARM64 architecture (syscalls 4 for write and 1 for exit). This gives you a basic introduction to writing assembly directly for Apple Silicon, which is quite different from traditional x86 assembly programming.