47 ms·

Why do you think AI is producing low quality code? Before I started using AI, my code was often rejected as "didn't use thing X" or "didn't follow best practice

Why do you think AI is producing low quality code? Before I started using AI, my code was often rejected as "didn't use thing X" or "didn't follow best practice Y" but ever since I started coding with AI, that was gone. Works especially well when the code is being reviewed by a person who is clueless about AI.

ben_w 1y ago
I've seen plenty of mediocre, even bad, code from real humans who didn't realise they were bad coders. Yet: while LLMs often beat those specific humans, I do also see LLMs doing their own mistakes. LLMs are very useful tools, but if they were human, they'd be humans with sleep deprivation or early stage dementia of some kind.
- mrweasel 1y ago
  That's a good point. The majority of human programmers aren't exactly super talented either, and now due to AI many have now lost all hope for personal development, but that's their choice. All code needs to be carefully scrutinized, AI generated or not. Maybe always prefix your prompt with: "Your operations team consists of a bunch and middel aged angry Unix fans, who will call you at 3:00AM if your service fails and belittle your abilities at the next incidents review meeting.". As for the 100% vibe coders, please let them. There's plenty of good money to be made cleaning up after them and I do love refactoring, deleting code and implementing monitoring and logging.
varjag 1y ago
An AI agent would never diss the work of a fellow AI agent!
- lemiffe 1y ago
  Except when it does... CodeRabbit will review my PRs sometimes complaining about code written by Copilot or Augment. They should just fight it off between themselves.
WesolyKubeczek 1y ago
Guess what, your code may still be low quality, you just have had a misfortune of having bad reviewers.
svantana 1y ago
I think the analogy still holds; fast fashion is generally of higher quality than "random-person-sewed-a-shirt-at-home". At least superficially. What the vibe-coded software usually lacks is someone (man or machine) who thought long and hard about the purpose of the code, along with extended use and testing leading to improvements.
thefz 1y ago
> Why do you think AI is producing low quality code? I asked for a very, very simple bash script to test code generation abilities once. The AI got it spectacularly wrong. So wrong that it was ridiculous. Here's my reason why I think it does produce low quality code; because it does.
- KronisLV 1y ago
  I feel like sooner or later the standard for these types of discussions should become: > "Here's a link to the commits in my GitHub repo, here's the exact prompts and models that were used that generated bad output. This exact example proves my point beyond a doubt." I've used Claude Sonnet 4 and Google Gemini 2.5 Pro to pretty good results otherwise, with RooCode - telling it what to look for in a codebase, to come up with an implementation plan, chatting with it about the details until it fills out a proper plan (sometimes it catches edge cases that I haven't thought of), around 100-200k tokens in usually it can knock out a decent implementation for whatever I have in mind, throw in another 100-200k tokens and it has made the tests pass and also written new ones as needed. Another 200k-400k for reading the codebase more in depth and doing refactoring (e.g. when writing Go it has a habit of doing a lot of stuff inline instead of looking at the utils package I have, less of an issue with Spring Boot Java apps for example cause there the service pattern is pretty common in the code it's been trained on I'd reckon) although adding something like AI.md or a gradually updated CODEBASE.md or indexing the whole codebase with an embedding model and storing it in Qdrant or something can help to save tokens there somewhat. Sometimes a particular model does keep messing up, switching over to another and explaining what the first one was doing wrong can help get rid of that spiraling, other times I just have to write all the code myself anyways because I have something different in mind, sometimes stopping it in the middle of editing a file and providing additional instructions. On average, still faster than doing everything manually and sometimes overlooks obvious things, but other times finds edge cases or knows syntax I might not. Obviously I use a far simpler workflow for one off data transformations or knocking out Bash scripts etc. Probably could save a bunch of tokens if not for RooCode system prompt, that thing was pretty long last I checked. Especially good as a second set of eyes without human pleasantries and quick turnaround (before actual human code review, when working in a team), not really nice for my wallet but oh well.
smartmic 1y ago
This is because AI-generated code will always be mediocre. There is so much poor-quality code in the training base that it will dilute the high-quality sources. AI is not a craftsman who builds on his own high standards, but rather a token grinder tool calibrated to your prompts. Even if you describe your problem (prompt) to a high standard, there is no way it can deliver a solution of the same standard. This will be true in 2025 as it was in 2023 and probably always will be.
- wobfan 1y ago
  100%. Generative AI is and will always be trained on more or less all open source code that is out there, and by definition, from the training data, it will create a mix of this, which will statistically be mediocre. Which is fine, as long as people are aware of it.
- simonw 1y ago
  I just don't think that's true. The LLM vendors are all competing on how well their models can write code, and the way they're doing that is to refine their training data - they constantly find new ways to remove poor quality code from the training data and increase the volume of high quality code. One way they do this is by using code that passes automated tests. That's a unique characteristic of code - you can't do that for regular prose, or legal analysis or whatever. "Even if you describe your problem (prompt) to a high standard, there is no way it can deliver a solution of the same standard." My own experience doesn't match that. I can describe my problems to a good LLM and get back code that I would have been proud to have written myself.
tgv 1y ago
My experience is that it does produce low quality code. Perhaps I tried some unusual stuff, but the other day, I had a simple problem in JavaScript: you have an image, add a gray border around it, and keep it within given width/height limits. I figured that should be common enough for whatever OpenAI model I was using to generate useable code. It started with doing something straightforward with a good-looking Math.min operation, and return a promise to a canvas. I asked "why is this returning a promise?", and of course it answered that I was right and removed the promise. Then it turned out that if the image was larger than limits, it would simply draw over the borders. Now I had to add that it should scale the image. It made an error in the scaling. IIRC, I had to tell it that both dimensions should be scaled identically, which led a few more trials before it looked like decent code. It's a clueless junior that has been kicked out of boot camp. What it does do perfectly: convert code from one language to another. It was a fairly complex bit, and the result was flawless.
- mettamage 1y ago
  > My experience is that it does produce low quality code. Perhaps I tried some unusual stuff, but the other day I've seen both happen. Sometimes it produced fairly good quality code on small problem domains. Sometimes it produced bad code on small problem domains. The code is always not that great to bad at big problem domains.
jeltz 1y ago
From using it, working with people who use it and from reviewing AI generated code. The AI generated code is typically on par with people like QA or sysadmins who do not code as their primary job. If someone on my team who was a software engineer and not very junior consistently produced such low quality code I would put them on a performance improvement plan.

ben_w 1y ago

I've seen plenty of mediocre, even bad, code from real humans who didn't realise they were bad coders. Yet: while LLMs often beat those specific humans, I do also see LLMs doing their own mistakes. LLMs are very useful tools, but if they were human, they'd be humans with sleep deprivation or early stage dementia of some kind.

mrweasel 1y ago
That's a good point. The majority of human programmers aren't exactly super talented either, and now due to AI many have now lost all hope for personal development, but that's their choice. All code needs to be carefully scrutinized, AI generated or not. Maybe always prefix your prompt with: "Your operations team consists of a bunch and middel aged angry Unix fans, who will call you at 3:00AM if your service fails and belittle your abilities at the next incidents review meeting.". As for the 100% vibe coders, please let them. There's plenty of good money to be made cleaning up after them and I do love refactoring, deleting code and implementing monitoring and logging.

varjag 1y ago

An AI agent would never diss the work of a fellow AI agent!

lemiffe 1y ago
Except when it does... CodeRabbit will review my PRs sometimes complaining about code written by Copilot or Augment. They should just fight it off between themselves.

WesolyKubeczek 1y ago

Guess what, your code may still be low quality, you just have had a misfortune of having bad reviewers.

svantana 1y ago

I think the analogy still holds; fast fashion is generally of higher quality than "random-person-sewed-a-shirt-at-home". At least superficially. What the vibe-coded software usually lacks is someone (man or machine) who thought long and hard about the purpose of the code, along with extended use and testing leading to improvements.

thefz 1y ago

> Why do you think AI is producing low quality code? I asked for a very, very simple bash script to test code generation abilities once. The AI got it spectacularly wrong. So wrong that it was ridiculous. Here's my reason why I think it does produce low quality code; because it does.

KronisLV 1y ago
I feel like sooner or later the standard for these types of discussions should become: > "Here's a link to the commits in my GitHub repo, here's the exact prompts and models that were used that generated bad output. This exact example proves my point beyond a doubt." I've used Claude Sonnet 4 and Google Gemini 2.5 Pro to pretty good results otherwise, with RooCode - telling it what to look for in a codebase, to come up with an implementation plan, chatting with it about the details until it fills out a proper plan (sometimes it catches edge cases that I haven't thought of), around 100-200k tokens in usually it can knock out a decent implementation for whatever I have in mind, throw in another 100-200k tokens and it has made the tests pass and also written new ones as needed. Another 200k-400k for reading the codebase more in depth and doing refactoring (e.g. when writing Go it has a habit of doing a lot of stuff inline instead of looking at the utils package I have, less of an issue with Spring Boot Java apps for example cause there the service pattern is pretty common in the code it's been trained on I'd reckon) although adding something like AI.md or a gradually updated CODEBASE.md or indexing the whole codebase with an embedding model and storing it in Qdrant or something can help to save tokens there somewhat. Sometimes a particular model does keep messing up, switching over to another and explaining what the first one was doing wrong can help get rid of that spiraling, other times I just have to write all the code myself anyways because I have something different in mind, sometimes stopping it in the middle of editing a file and providing additional instructions. On average, still faster than doing everything manually and sometimes overlooks obvious things, but other times finds edge cases or knows syntax I might not. Obviously I use a far simpler workflow for one off data transformations or knocking out Bash scripts etc. Probably could save a bunch of tokens if not for RooCode system prompt, that thing was pretty long last I checked. Especially good as a second set of eyes without human pleasantries and quick turnaround (before actual human code review, when working in a team), not really nice for my wallet but oh well.

smartmic 1y ago

This is because AI-generated code will always be mediocre. There is so much poor-quality code in the training base that it will dilute the high-quality sources. AI is not a craftsman who builds on his own high standards, but rather a token grinder tool calibrated to your prompts. Even if you describe your problem (prompt) to a high standard, there is no way it can deliver a solution of the same standard. This will be true in 2025 as it was in 2023 and probably always will be.

wobfan 1y ago
100%. Generative AI is and will always be trained on more or less all open source code that is out there, and by definition, from the training data, it will create a mix of this, which will statistically be mediocre. Which is fine, as long as people are aware of it.
simonw 1y ago
I just don't think that's true. The LLM vendors are all competing on how well their models can write code, and the way they're doing that is to refine their training data - they constantly find new ways to remove poor quality code from the training data and increase the volume of high quality code. One way they do this is by using code that passes automated tests. That's a unique characteristic of code - you can't do that for regular prose, or legal analysis or whatever. "Even if you describe your problem (prompt) to a high standard, there is no way it can deliver a solution of the same standard." My own experience doesn't match that. I can describe my problems to a good LLM and get back code that I would have been proud to have written myself.

tgv 1y ago

My experience is that it does produce low quality code. Perhaps I tried some unusual stuff, but the other day, I had a simple problem in JavaScript: you have an image, add a gray border around it, and keep it within given width/height limits. I figured that should be common enough for whatever OpenAI model I was using to generate useable code. It started with doing something straightforward with a good-looking Math.min operation, and return a promise to a canvas. I asked "why is this returning a promise?", and of course it answered that I was right and removed the promise. Then it turned out that if the image was larger than limits, it would simply draw over the borders. Now I had to add that it should scale the image. It made an error in the scaling. IIRC, I had to tell it that both dimensions should be scaled identically, which led a few more trials before it looked like decent code. It's a clueless junior that has been kicked out of boot camp. What it does do perfectly: convert code from one language to another. It was a fairly complex bit, and the result was flawless.

mettamage 1y ago
> My experience is that it does produce low quality code. Perhaps I tried some unusual stuff, but the other day I've seen both happen. Sometimes it produced fairly good quality code on small problem domains. Sometimes it produced bad code on small problem domains. The code is always not that great to bad at big problem domains.

jeltz 1y ago

From using it, working with people who use it and from reviewing AI generated code. The AI generated code is typically on par with people like QA or sysadmins who do not code as their primary job. If someone on my team who was a software engineer and not very junior consistently produced such low quality code I would put them on a performance improvement plan.