4 ms·

Can you not review it?

throwaway173738 5mo ago
Review always misses something.
RugnirViking 5mo ago
I think this misses the scale of the problem. Review never fixed tech debt, nor did it fix relevant/bloated test suites. It didn't solve complexity, or eliminate footguns. Very few people (I would argue almost noone) had developed theories for what all of these even were, or how to spot them in code. Reviewers aren't perfect, far from it. And we just gave them ~20x more code to review. Incentives mean that taking 20x longer to review is unacceptable. So where do we go from here?
- mountainriver 5mo ago
  Certainly not back to hand crafted code unless you want to be left in the dust
bigstrat2003 5mo ago
You can review it, but that negates any productivity gains from using the LLM in the first place.
- dkersten 5mo ago
  Agreed. Reviewing code takes so much longer and is far more exhausting than writing it, and you still don’t understand the logic as well or intuitively as you would if you write it.
  - runeb 5mo ago
    Code reviews should be done by someone other than the author though, so the only thing that changes with ai generated code in that respect is the amount of it
    - dkersten 5mo ago
      That doesn’t conflict with my statement. Before: One person writes the code (and likely understands it thoroughly), another person reviews the code to spot obvious mistakes or shortcomings. Now: AI writes the code, a person reviews it to spot obvious mistakes or shortcomings. In the before case, you have a person who has a deeper understanding of the code and in the AI case, you don’t, instead you have even more code to review. When a competent programmer is writing the code, the human written code tends to be higher quality too. So it’s not just about review quantity but the quality of code being reviewed. Some people claim the AI writes great code, but that just hasn’t been my experience yet (at least with the models I’ve tried, including Opus). They still make ridiculously bad decisions regularly.
      - mountainriver 5mo ago
        >When a competent programmer is writing the code, the human written code tends to be higher quality too This is a great idea, but on average is deeply untrue. Far and away most programmers today write significantly worse code than LLMs. Also LLMs are fantastic at generating high level summaries and comments in code
        dkersten 5mo ago
        > Far and away most programmers today write significantly worse code than LLMs Your experience with LLMs do not match my own. Not to say that I haven’t experienced terrible human written code where I’ve wondered what the author could possibly have been thinking, but overall, I still find LLM written code to be on the poor side. Like, the code itself is ok, but the wider picture reasoning and abstractions are bad. It also makes really dumb decisions far too often. Or doggedly shoehorns its first idea in no matter how badly it fits.
duskdozer 5mo ago
I'm really amused at how quickly things changed from "yes, the AI is writing it all, but I'm carefully reviewing every line" to "it would take too long and be too confusing to review any of it"

Review always misses something.

I think this misses the scale of the problem. Review never fixed tech debt, nor did it fix relevant/bloated test suites. It didn't solve complexity, or eliminate footguns. Very few people (I would argue almost noone) had developed theories for what all of these even were, or how to spot them in code. Reviewers aren't perfect, far from it. And we just gave them ~20x more code to review. Incentives mean that taking 20x longer to review is unacceptable. So where do we go from here?

mountainriver 5mo ago
Certainly not back to hand crafted code unless you want to be left in the dust

bigstrat2003 5mo ago

You can review it, but that negates any productivity gains from using the LLM in the first place.

dkersten 5mo ago
Agreed. Reviewing code takes so much longer and is far more exhausting than writing it, and you still don’t understand the logic as well or intuitively as you would if you write it.
- runeb 5mo ago
  Code reviews should be done by someone other than the author though, so the only thing that changes with ai generated code in that respect is the amount of it
  - dkersten 5mo ago
    That doesn’t conflict with my statement. Before: One person writes the code (and likely understands it thoroughly), another person reviews the code to spot obvious mistakes or shortcomings. Now: AI writes the code, a person reviews it to spot obvious mistakes or shortcomings. In the before case, you have a person who has a deeper understanding of the code and in the AI case, you don’t, instead you have even more code to review. When a competent programmer is writing the code, the human written code tends to be higher quality too. So it’s not just about review quantity but the quality of code being reviewed. Some people claim the AI writes great code, but that just hasn’t been my experience yet (at least with the models I’ve tried, including Opus). They still make ridiculously bad decisions regularly.
    - mountainriver 5mo ago
      >When a competent programmer is writing the code, the human written code tends to be higher quality too This is a great idea, but on average is deeply untrue. Far and away most programmers today write significantly worse code than LLMs. Also LLMs are fantastic at generating high level summaries and comments in code
      - dkersten 5mo ago
        > Far and away most programmers today write significantly worse code than LLMs Your experience with LLMs do not match my own. Not to say that I haven’t experienced terrible human written code where I’ve wondered what the author could possibly have been thinking, but overall, I still find LLM written code to be on the poor side. Like, the code itself is ok, but the wider picture reasoning and abstractions are bad. It also makes really dumb decisions far too often. Or doggedly shoehorns its first idea in no matter how badly it fits.

duskdozer 5mo ago

I'm really amused at how quickly things changed from "yes, the AI is writing it all, but I'm carefully reviewing every line" to "it would take too long and be too confusing to review any of it"