3 ms·

Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code. Trust me, I've seen enough PRs to last a life time. A lot

by malfist 13d ago

Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code. Trust me, I've seen enough PRs to last a life time. A lot of professionals wouldn't know good code if it slapped them in the face.

asutekku 13d ago
I'd argue for most people LLM produces much better code than they would be able to write themselves.
- malfist 13d ago
  That is not an argument that LLMs produce good code
  - bluGill 13d ago
    They produce good code when I'm personally reviewing them. There are a few other people who work with who likewise know how to review code and thus can get good code out of an LLM. There are, however, a lot of people who just accept the first slop that they get out of it and that's not good code. The larger issue of good code isn't the actual individual lines, it's the overall architecture. And that's what I'm going to be reviewing first is, is this a good approach? Then the interfaces to other code is this a good interface. Get those two right and we can go back for the details. In a lot of cases, the LLM is plenty good at those details. In some cases, an LLM is better than what I could do. Well, I suppose I can trace down all the locks in all the different special cases, and I have done that, but that was a huge amount of effort that I really don't want to repeat. Note that I'm talking about recent models. If you're asking about the models of just one year ago, I would give a very different answer about the type of code an LLM produces.
- abalashov 12d ago
  I'm not sure how literally you mean "most people". This might be true in a purely volumetric sense, but that's not really the bar around these parts...
preg_match 12d ago
You can most definitely produce high-quality code via an LLM, particularly if you test your code aggressively and review it. Yes there are a lot of shoddy PRs, but that's nothing too new. The problem with LLMs is the amount of code they produce. More code = more garbage. But, that code velocity can be leveraged to increase quality. Through careful design and testing. Ultimately, I would take LLM code + high-quality multi-strategy testing over human code with little to no tests. And some would say "well that's a false dichotomy". I disagree, before LLMs engineers didn't have the time or incentives to aggressively test. The tests either would not exist, or would be shitty unit tests intended to get an arbitrary coverage percentage. Now, we can write high-quality tests, differential testing, fuzzing, and more, in much less time.
- rented_mule 12d ago
  Going much deeper on tests has been transformative for me. In a solo project started from scratch, I'm 6-7 weeks in, and it's up to ~90K lines. ~60K of those are tests. Those tests have now found (and then the agent has correctly diagnosed) multiple bugs in broadly used libraries that I'm using in my project. That's because those bugs surfaced as occasional issues in my project. Especially powerful are all the property-based tests (perhaps what you are calling fuzzing? I'm using the Python package called Hypothesis for this). Another spectrum that I've found useful to explore is the scope of what I ask the coding agent to do in one turn. I see some people trying to do one massive prompt that the coding agent works on for a day or more. I find a large boost in overall quality if I do 10-20 prompts per day (not counting the prompts where I'm just trying to understand things). It's still much less of my time than hand-coding, but the resulting architecture looks like my own. The quality of the overall system is great. There are certainly issues here and there in the code, but it's always that way once a project gets large enough. Now it's easier to address any particular issue throughout the code base in one go.
deterministic 12d ago
> Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code Sorry, but you are 100% wrong. I have 30+ years of professional development experience working on complex, very large-scale C++ code used by companies around the world. I care deeply about code quality and always have. More than any other developer I've worked with in my 30+ year career. And I'm now using Claude Code to push the quality bar much higher. But you have to learn how to use it properly. It's a tool. Quality doesn't happen automatically. It's a big mistake, and frankly quite arrogant, to assume that because it doesn't work for you, it can't work for anyone else. Or that the rest of us must either be lying or incompetent.
- player1234 12d ago
  [dead]
- blub 12d ago
  It’s fair to ask people bragging about their amazing AI skills to show their code or GTFO. Hope it becomes established.

by malfist 13d ago

asutekku 13d ago

I'd argue for most people LLM produces much better code than they would be able to write themselves.

malfist 13d ago
That is not an argument that LLMs produce good code
- bluGill 13d ago
  They produce good code when I'm personally reviewing them. There are a few other people who work with who likewise know how to review code and thus can get good code out of an LLM. There are, however, a lot of people who just accept the first slop that they get out of it and that's not good code. The larger issue of good code isn't the actual individual lines, it's the overall architecture. And that's what I'm going to be reviewing first is, is this a good approach? Then the interfaces to other code is this a good interface. Get those two right and we can go back for the details. In a lot of cases, the LLM is plenty good at those details. In some cases, an LLM is better than what I could do. Well, I suppose I can trace down all the locks in all the different special cases, and I have done that, but that was a huge amount of effort that I really don't want to repeat. Note that I'm talking about recent models. If you're asking about the models of just one year ago, I would give a very different answer about the type of code an LLM produces.
abalashov 12d ago
I'm not sure how literally you mean "most people". This might be true in a purely volumetric sense, but that's not really the bar around these parts...

preg_match 12d ago

You can most definitely produce high-quality code via an LLM, particularly if you test your code aggressively and review it. Yes there are a lot of shoddy PRs, but that's nothing too new. The problem with LLMs is the amount of code they produce. More code = more garbage. But, that code velocity can be leveraged to increase quality. Through careful design and testing. Ultimately, I would take LLM code + high-quality multi-strategy testing over human code with little to no tests. And some would say "well that's a false dichotomy". I disagree, before LLMs engineers didn't have the time or incentives to aggressively test. The tests either would not exist, or would be shitty unit tests intended to get an arbitrary coverage percentage. Now, we can write high-quality tests, differential testing, fuzzing, and more, in much less time.

rented_mule 12d ago
Going much deeper on tests has been transformative for me. In a solo project started from scratch, I'm 6-7 weeks in, and it's up to ~90K lines. ~60K of those are tests. Those tests have now found (and then the agent has correctly diagnosed) multiple bugs in broadly used libraries that I'm using in my project. That's because those bugs surfaced as occasional issues in my project. Especially powerful are all the property-based tests (perhaps what you are calling fuzzing? I'm using the Python package called Hypothesis for this). Another spectrum that I've found useful to explore is the scope of what I ask the coding agent to do in one turn. I see some people trying to do one massive prompt that the coding agent works on for a day or more. I find a large boost in overall quality if I do 10-20 prompts per day (not counting the prompts where I'm just trying to understand things). It's still much less of my time than hand-coding, but the resulting architecture looks like my own. The quality of the overall system is great. There are certainly issues here and there in the code, but it's always that way once a project gets large enough. Now it's easier to address any particular issue throughout the code base in one go.

deterministic 12d ago

> Anyone who thinks they can produce high quality code from an LLM is mistaken about how to judge code Sorry, but you are 100% wrong. I have 30+ years of professional development experience working on complex, very large-scale C++ code used by companies around the world. I care deeply about code quality and always have. More than any other developer I've worked with in my 30+ year career. And I'm now using Claude Code to push the quality bar much higher. But you have to learn how to use it properly. It's a tool. Quality doesn't happen automatically. It's a big mistake, and frankly quite arrogant, to assume that because it doesn't work for you, it can't work for anyone else. Or that the rest of us must either be lying or incompetent.

player1234 12d ago
[dead]
blub 12d ago
It’s fair to ask people bragging about their amazing AI skills to show their code or GTFO. Hope it becomes established.