4 ms·

In my experience (with ChatGPT 5.1 as of late) is that the AI follows a problem->solution internal logic and doesn't think and try to structure its code. If yo

by torginus 11mo ago

In my experience (with ChatGPT 5.1 as of late) is that the AI follows a problem->solution internal logic and doesn't think and try to structure its code. If you ask for an endpoint to a CRUD API, it'll make one. If you ask for 5, it'll repeat the same code 5 times and modify it for the use case. A dev wouldn't do this, they would try to figure out the common parts of code, pull them out into helpers, and try to make as little duplicated code as possible. I feel like the AI has a strong bias towards adding things, and not removing them. The most obviously wrong thing is with CSS - when I try to do some styling, it gets 90% of the way there, but there's almost always something that's not quite right. Then I tell the AI to fix a style, since that div is getting clipped or not correctly centered etc. It almost always keeps adding properties, and after 2-3 tries and an incredibly bloated style, I delete the thing and take a step back and think logically about how to properly lay this out with flexbox.

BurningFrog 11mo ago
Bad code is only really bad if it needs to be maintained. If your AI reliably generates working code from a detailed prompt, the prompt is now the source that needs to be maintained. There is no important reason to even look at the generated code
- AlexandrB 11mo ago
  I think this might be plausible in the future, but it needs a lot more tooling. For starters you need to be able to run the prompt through the exact same model so you can reproduce a "build".
  - josephg 11mo ago
    Even the exact same model isn't enough. There are several sources of nondeterminism in LLMs. These would all need to be squashed or seeded - which as far as I know isn't a feature that openai / anthropic / etc provide.
  - BurningFrog 11mo ago
    OK, then the current models aren't as good as I thought/hoped. I guess one thing it means is that we still need extensive test suites. I suppose an LLM can write those too.
- josephg 11mo ago
  Well.. except the AI models are nondeterministic. If you ask an AI the same prompt 20 times, you'll get 20 different answers. Some of them might work, some probably won't. It usually takes a human to tell which are which and fix problems & refactor. If you keep the prompt, you can't manually modify the generated code afterwards (since it'll be regenerated). Even if you get the AI to write all the code correctly, there's no guarantee it'll do the same thing next time.
- anon7725 11mo ago
  > the prompt is now the source that needs to be maintained The inference response to the prompt is not deterministic. In fact, it’s probably chaotic since small changes to the prompt can produce large changes to the inference.
  - CamperBob2 11mo ago
    The inference response to the prompt is not deterministic. So? Nobody cares. Is the output of your C compiler the same every time you run it? How about your FPGA synthesis tool? Is that deterministic? Are you sure? What difference does it make, as long as the code works?
    - josephg 11mo ago
      The C compiler will still make working programs every time, so long as your code isn’t broken. But sometimes the code chatgpt produces won’t work. Or it'll kinda work but you’ll get weird, different bugs each time you generate it. No thanks.
      - CamperBob2 11mo ago
        Nothing matters but d/dt. It's so much better than it was a year ago, it's not even funny. How weird would it be if something like this worked perfectly out of the box, with no need for further improvement and refinement?
    - antiloper 11mo ago
      > Is the output of your C compiler the same every time you run it? Yes? Because of actual engineering mind you and not rolling the dice until the lucky number comes up. https://reproducibility.nixos.social/evaluations/2/2d293cbfa5a7 https://reproducibility.nixos.social/evaluations/2/2d293cbfa...
      - CamperBob2 11mo ago
        It's not true for a place-and-route engine, so why does it have to be true for a C compiler? Nobody else cares. If you do, that's great, I guess... but you'll be outcompeted by people who don't.
bigblind 11mo ago
> If you ask for an endpoint to a CRUD API, it'll make one. If you ask for 5, it'll repeat the same code 5 times and modify it for the use case. > >A dev wouldn't do this, they would try to figure out the common parts of code, pull them out into helpers, and try to make as little duplicated code as possible. > >I feel like the AI has a strong bias towards adding things, and not removing them. I suspect this is because an LLM doesn't build a mental model of the code base like a dev does. It can decide to look at certain files, and maybe you can improve this by putting a broad architecture overview of a system in an agents.md file, I don't have much experience with that. But for now, I'm finding it most useful still think in terms of code architecture, and give it small steps that are part of that architecture, and then iterate based on your own review of AI generated code. I don't have the confidence in it to just let some agent plan, and then run for tens of minutes or even hours building out a feature. I want to be in the loop earlier to set the direction.
skissane 11mo ago
> If you ask for an endpoint to a CRUD API, it'll make one. If you ask for 5, it'll repeat the same code 5 times and modify it for the use case. I don’t think this is an inherent issue to the technology. Duplicate code detectors have been around for ages. Given an AI agent a tool which calls one, and ask it to reduce duplication, it will start refactoring. Of course, there is a risk of going too far in the other direction-refactorings which technically reduce duplication but which have unacceptable costs (you can be too DRY). But some possible solutions: (a) ask it to judge if the refactoring is worth it or not - if it judges no, just ignore the duplication and move on; (b) get a human to review the decision in (a); (c) if AI repeatedly makes wrong decision (according to human), prompt engineering, or maybe even just some hardcoded heuristics
- dasil003 11mo ago
  Sorry but from where I sit, this is only marginally closes gap from AI to truly senior engineers. Basically human junior engineers start by writing code in a very procedural and literal style with duplicate logic all over the place because that's the first step in adapting human intelligence to learning how to program. Then the programmer realizes this leads to things becoming unmaintainable and so they start to learn the abstraction techniques of functions, etc. An LLM doesn't have to learn any of that, because they already know all languages and mechanical technique in their corpus, so this beginning journey never applies. But what the junior programmer has that the LLM doesn't, is an innate common sense understanding of human goals that are driving the creation of the code to begin with, and that serves them through their entire progression from junior to senior. As you point out, code can be "too DRY", but why? Senior engineers understand that DRYing up code is not a style issue, its more about maintainability and understanding what is likely to change, and what will be the apparent effects to human stakeholders who depend on the software. Basically do these things map to things that are conceptually the same for human users and are unlikely to diverge in the future. This is also a surprisingly deep question as perhaps every human stakeholder will swear up and down they are the same, but nevertheless 6 months from now a problem arises that requires them to diverge. At this point there is now a cognitive overhead and dissonance of explaining that divergence of the users who were heretofore perfectly satisfied with one domain concept. Ultimately the value function for success of a specific code factoring style depends on a lot of implicit context and assumptions that are baked into the heads of various stakeholders for the specific use case and can change based on myriad outside factors that are not visible to an LLM. Senior engineers understand the map is not the territory, for LLMs there is no territory.
babyshake 11mo ago
A good system prompt goes a long way with the latest models. Even just something as simple as "use DRY principles whenever possible." or prompting a plan-implement-evaluate cycle gets pretty good results, at least for tasks that are doing things that AI is well trained on like CRUD APIs.
BenkaiDebussy 11mo ago
I've noticed this as well, though I've also noticed that you can sometimes avoid it if you're more explicit and actually say things like "can you write these endpoints in a way that ___, ___, and ____." Or I'll mention some context that I'm worried the LLM will miss (for example pointing out when there are already existing functions for doing certain things). The broader a request is, the more likely I am to get a bunch of bloat. I think this is partly because the LLM will always try to fully solve the problem entirely from your initial prompt. Instead of stopping to clarify something, it'll just move forward with doing something that technically works. I find it's better to break things into smaller steps so that you can "intervene" if it starts to do things wrong.

BurningFrog 11mo ago

Bad code is only really bad if it needs to be maintained. If your AI reliably generates working code from a detailed prompt, the prompt is now the source that needs to be maintained. There is no important reason to even look at the generated code

AlexandrB 11mo ago
I think this might be plausible in the future, but it needs a lot more tooling. For starters you need to be able to run the prompt through the exact same model so you can reproduce a "build".
- josephg 11mo ago
  Even the exact same model isn't enough. There are several sources of nondeterminism in LLMs. These would all need to be squashed or seeded - which as far as I know isn't a feature that openai / anthropic / etc provide.
- BurningFrog 11mo ago
  OK, then the current models aren't as good as I thought/hoped. I guess one thing it means is that we still need extensive test suites. I suppose an LLM can write those too.
josephg 11mo ago
Well.. except the AI models are nondeterministic. If you ask an AI the same prompt 20 times, you'll get 20 different answers. Some of them might work, some probably won't. It usually takes a human to tell which are which and fix problems & refactor. If you keep the prompt, you can't manually modify the generated code afterwards (since it'll be regenerated). Even if you get the AI to write all the code correctly, there's no guarantee it'll do the same thing next time.
anon7725 11mo ago
> the prompt is now the source that needs to be maintained The inference response to the prompt is not deterministic. In fact, it’s probably chaotic since small changes to the prompt can produce large changes to the inference.
- CamperBob2 11mo ago
  The inference response to the prompt is not deterministic. So? Nobody cares. Is the output of your C compiler the same every time you run it? How about your FPGA synthesis tool? Is that deterministic? Are you sure? What difference does it make, as long as the code works?
  - josephg 11mo ago
    The C compiler will still make working programs every time, so long as your code isn’t broken. But sometimes the code chatgpt produces won’t work. Or it'll kinda work but you’ll get weird, different bugs each time you generate it. No thanks.
    - CamperBob2 11mo ago
      Nothing matters but d/dt. It's so much better than it was a year ago, it's not even funny. How weird would it be if something like this worked perfectly out of the box, with no need for further improvement and refinement?
  - antiloper 11mo ago
    > Is the output of your C compiler the same every time you run it? Yes? Because of actual engineering mind you and not rolling the dice until the lucky number comes up. https://reproducibility.nixos.social/evaluations/2/2d293cbfa5a7 https://reproducibility.nixos.social/evaluations/2/2d293cbfa...
    - CamperBob2 11mo ago
      It's not true for a place-and-route engine, so why does it have to be true for a C compiler? Nobody else cares. If you do, that's great, I guess... but you'll be outcompeted by people who don't.

bigblind 11mo ago

> If you ask for an endpoint to a CRUD API, it'll make one. If you ask for 5, it'll repeat the same code 5 times and modify it for the use case. > >A dev wouldn't do this, they would try to figure out the common parts of code, pull them out into helpers, and try to make as little duplicated code as possible. > >I feel like the AI has a strong bias towards adding things, and not removing them. I suspect this is because an LLM doesn't build a mental model of the code base like a dev does. It can decide to look at certain files, and maybe you can improve this by putting a broad architecture overview of a system in an agents.md file, I don't have much experience with that. But for now, I'm finding it most useful still think in terms of code architecture, and give it small steps that are part of that architecture, and then iterate based on your own review of AI generated code. I don't have the confidence in it to just let some agent plan, and then run for tens of minutes or even hours building out a feature. I want to be in the loop earlier to set the direction.

skissane 11mo ago

> If you ask for an endpoint to a CRUD API, it'll make one. If you ask for 5, it'll repeat the same code 5 times and modify it for the use case. I don’t think this is an inherent issue to the technology. Duplicate code detectors have been around for ages. Given an AI agent a tool which calls one, and ask it to reduce duplication, it will start refactoring. Of course, there is a risk of going too far in the other direction-refactorings which technically reduce duplication but which have unacceptable costs (you can be too DRY). But some possible solutions: (a) ask it to judge if the refactoring is worth it or not - if it judges no, just ignore the duplication and move on; (b) get a human to review the decision in (a); (c) if AI repeatedly makes wrong decision (according to human), prompt engineering, or maybe even just some hardcoded heuristics

dasil003 11mo ago
Sorry but from where I sit, this is only marginally closes gap from AI to truly senior engineers. Basically human junior engineers start by writing code in a very procedural and literal style with duplicate logic all over the place because that's the first step in adapting human intelligence to learning how to program. Then the programmer realizes this leads to things becoming unmaintainable and so they start to learn the abstraction techniques of functions, etc. An LLM doesn't have to learn any of that, because they already know all languages and mechanical technique in their corpus, so this beginning journey never applies. But what the junior programmer has that the LLM doesn't, is an innate common sense understanding of human goals that are driving the creation of the code to begin with, and that serves them through their entire progression from junior to senior. As you point out, code can be "too DRY", but why? Senior engineers understand that DRYing up code is not a style issue, its more about maintainability and understanding what is likely to change, and what will be the apparent effects to human stakeholders who depend on the software. Basically do these things map to things that are conceptually the same for human users and are unlikely to diverge in the future. This is also a surprisingly deep question as perhaps every human stakeholder will swear up and down they are the same, but nevertheless 6 months from now a problem arises that requires them to diverge. At this point there is now a cognitive overhead and dissonance of explaining that divergence of the users who were heretofore perfectly satisfied with one domain concept. Ultimately the value function for success of a specific code factoring style depends on a lot of implicit context and assumptions that are baked into the heads of various stakeholders for the specific use case and can change based on myriad outside factors that are not visible to an LLM. Senior engineers understand the map is not the territory, for LLMs there is no territory.

babyshake 11mo ago

A good system prompt goes a long way with the latest models. Even just something as simple as "use DRY principles whenever possible." or prompting a plan-implement-evaluate cycle gets pretty good results, at least for tasks that are doing things that AI is well trained on like CRUD APIs.

BenkaiDebussy 11mo ago

I've noticed this as well, though I've also noticed that you can sometimes avoid it if you're more explicit and actually say things like "can you write these endpoints in a way that ___, ___, and ____." Or I'll mention some context that I'm worried the LLM will miss (for example pointing out when there are already existing functions for doing certain things). The broader a request is, the more likely I am to get a bunch of bloat. I think this is partly because the LLM will always try to fully solve the problem entirely from your initial prompt. Instead of stopping to clarify something, it'll just move forward with doing something that technically works. I find it's better to break things into smaller steps so that you can "intervene" if it starts to do things wrong.