5 ms·

I am curious about this also. I have now seen multiple PR's that I had to review that a method was clearly completely modified by AI with no good reason and whe

I am curious about this also. I have now seen multiple PR's that I had to review that a method was clearly completely modified by AI with no good reason and when asked why something was changed we just got silence. Not exaggerating, literal silence and then trying to ignore the question and explain the thing we were first asking them to do. Clearly having no idea what is actually in this PR. This was done because we asked for a minor change to be done (talking maybe 5 lines of code) and tested. So now not only are we dealing with new debt, we are dealing with code that no one can explain why it was completely changed (and some of the changes were changes for the sake of change), and we are dealing with those of us that manage this code now looking at completely foreign code. I keep seeing this with people that are using these tools and they are not higher level engineers. We finally got to the point of denying these PR's and saying to go back and do it again. Loosing any of the time that was theoretically gained from doing it in the first place. Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base.

okwhateverdude 2y ago
> Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base. It is worse than that. We're all maintaining in our heads the mental sand castle that is the system the code base represents. The abuse of the autocoder erodes that sand castle because the intentions of the changes, which are crucial for mentally updating the sand castle, are not communicated (because they are unknowable). This is same thing with poor commit messages, or poor documentation around requirements/business processes. With enough erosion, plus expected turn over in staff, the sand castle is actually gone.
- rongenre 2y ago
  There needs to be a pattern for including AI-generated code, recording all the context which led to the generation.
  - jcgrillo 2y ago
    Easy: exclude developers who try it. Learning--be it a new codebase, a new programming language, a new database--takes time. You're not going to be as productive until you learn how. That's fine! Cheating on your homework with an LLM should not be something we celebrate, though, because the learner will never become productive that way, and they won't understand the code they're submitting for review.
    - XenophileJKO 2y ago
      The truth is that the tools are actually quite good already. If you know what you are doing they will 10-20x your productivity. Ultimately not adopting them will religate you to the same fate as assembly programmers. Sure there are place for it, but you won't be able to get near as much functionally done in the same amount of time and there won't be as much demand for it.
      - shinycode 2y ago
        Do you agree that the brain-memory activity of writing code and reading someone else’s code is totally different ? The sand castle analogy is still valid here because once you have a x10 productivity or worse a x20 one, there is no way you can deeply understand the things the same way than if you wrote it from scratch. Without spending a considerable amount of time and getting productivity down, the understanding is not the same. If no one is responsible because it’s crap software and you won’t be around enough time to bear responsibility… it’s ok I guess ?
        andai 2y ago
        I write at 1x speed and I can't even understand my own code the next day.
      - kragen 2y ago
        if you are seeing 900% productivity gains, why did these controlled experiments only find 28%, mostly among programmers who don't know what they're doing? and only 8–10% among programmers who did? do you have any hypotheses? i suspect you are seeing 900% productivity gains on certain narrow tasks (like greenfield prototype-quality code using apis you aren't familiar with) and incorrectly extrapolating to programming as a whole
jonnycomputer 2y ago
When using code assist, I've occasionally found some perplexing changes to my code I didn't remember making (and wouldn't have made). Can be pretty frustrating.

okwhateverdude 2y ago

> Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base. It is worse than that. We're all maintaining in our heads the mental sand castle that is the system the code base represents. The abuse of the autocoder erodes that sand castle because the intentions of the changes, which are crucial for mentally updating the sand castle, are not communicated (because they are unknowable). This is same thing with poor commit messages, or poor documentation around requirements/business processes. With enough erosion, plus expected turn over in staff, the sand castle is actually gone.

rongenre 2y ago
There needs to be a pattern for including AI-generated code, recording all the context which led to the generation.
- jcgrillo 2y ago
  Easy: exclude developers who try it. Learning--be it a new codebase, a new programming language, a new database--takes time. You're not going to be as productive until you learn how. That's fine! Cheating on your homework with an LLM should not be something we celebrate, though, because the learner will never become productive that way, and they won't understand the code they're submitting for review.
  - XenophileJKO 2y ago
    The truth is that the tools are actually quite good already. If you know what you are doing they will 10-20x your productivity. Ultimately not adopting them will religate you to the same fate as assembly programmers. Sure there are place for it, but you won't be able to get near as much functionally done in the same amount of time and there won't be as much demand for it.
    - shinycode 2y ago
      Do you agree that the brain-memory activity of writing code and reading someone else’s code is totally different ? The sand castle analogy is still valid here because once you have a x10 productivity or worse a x20 one, there is no way you can deeply understand the things the same way than if you wrote it from scratch. Without spending a considerable amount of time and getting productivity down, the understanding is not the same. If no one is responsible because it’s crap software and you won’t be around enough time to bear responsibility… it’s ok I guess ?
      - andai 2y ago
        I write at 1x speed and I can't even understand my own code the next day.
    - kragen 2y ago
      if you are seeing 900% productivity gains, why did these controlled experiments only find 28%, mostly among programmers who don't know what they're doing? and only 8–10% among programmers who did? do you have any hypotheses? i suspect you are seeing 900% productivity gains on certain narrow tasks (like greenfield prototype-quality code using apis you aren't familiar with) and incorrectly extrapolating to programming as a whole

jonnycomputer 2y ago

When using code assist, I've occasionally found some perplexing changes to my code I didn't remember making (and wouldn't have made). Can be pretty frustrating.