4 ms·

> AI has help me refactor things where I normally couldn’t. Reading "couldn't" as, you would technically not be able to do it because of the complexity or int

> AI has help me refactor things where I normally couldn’t. Reading "couldn't" as, you would technically not be able to do it because of the complexity or intricacy of the problem, how did you guarantee that the change offered by the AI made proper sense and didn't leave out critical patterns that were too complex for you to detect ? Your comment makes it sound like you're now dependent on AI to refactor again if dire consequences are detected way down the line (in a few months for instance), and the problem space is already just not graspable by a mere human. Which sounds really bad if that's the case.

firecall 1y ago
That's an important distinction! There's a difference between not intellectually understanding something and not being able to refactor something because if you start pulling on a thread, you are not sure what will unravel! And often there just isn't time allocated in a budget to begin an unlimited game of bug testing whack-a-mole!
- OccamsMirror 1y ago
  To makeitdouble's point, how is this any different with an LLM provided solution? What confidence do you have that isn't also beginning an unlimited game of bug testing whack-a-mole? My confidence in LLMs is not that high and I use Claude a lot. The limitations are very apparent very quickly. They're great for simple refactors and doing some busy work, but if you're refactoring something you're too afraid to do by hand then I fear you've simply deferred responsibility to the LLM - assuming it will understand the code better than you do, which seems foolhardy.
  - rocqua 1y ago
    Especially with refactoring, it tends to be tedious and repetitive work that is slightly too complicated for a (regex) search replace. A lot of repetitive slight variations on the same easy to describe change sounds pretty good to ask an LLM to do quickly.
pshc 1y ago
Sometimes a problem is a weird combination of hairy/obscure/tedious where I simply don’t have the activation energy to get started. Like, I could do it with a gun to my head. But if someone else were to do it for me I would gratefully review the merge request.
- strogonoff 1y ago
  Reviewing a merge request should require at least the same activation energy as writing the solution yourself, as in order to adequately evaluate a solution you first need to acquire a reference point in mind as to what the right solution should be in the first place. For me personally, the activation energy is higher when reviewing: it’s fun to come up with the solution that ends up being used, not so fun to come up with a solution that just serves as a reference point for evaluation and then gets immediately thrown away. Plus, I know in advance that a lot of cycles will be wasted on trying to understand how someone else’s vision maps onto my solution, especially when that vision is muddy.
  - solaire_oa 1y ago
    The submitter should also have thoroughly reviewed their own MR/PR. Even before LLMs, coders not having reviewed their own code would be completely discourteous and disrespectful to the reviewer. It's an embarrassing faux pas that makes the submitter and the team all look and feel bad when there are obvious problems that need to be called out and fixed. Submitting LLM barf for review and not reviewing it should be grounds for termination. The only way I can envision LLM barf being sustainable, or plausible, is if you removed code review altogether.
    - strogonoff 1y ago
      > The submitter should also have thoroughly reviewed their own MR/PR What does it mean to have to review your own code as a separate activity? Do many people contribute code that they wrote but… never read? > Submitting LLM barf Oh right…
      - 1y ago
Aeolun 1y ago
I think, if it’s similar to how I feel about it, that it’s more about always being able to do it, but not wanting to expend the mental effort to correctly adjust all those 30 places. Your boss is not going to care, so while it’s a bit better going forward, justifying the time to do it manually doesn’t make sense even to yourself. If you can do it using an LLM in a few hours however, suddenly making your life, and the lives of everyone that comes after you, easier becomes a pretty simple decision.
- djtango 1y ago
  So everyone is talking across each other... AI is a sharp tool, use it well and it cuts. Use it poorly and it'll cut you. Helping you overcome the activation barrier to make that redactor is great if that truly is what it is. That is probably still worth billions in the aggregate given git is considered billion dollar software. But slop piled on top of slop piled on top of slop is only going to compound all the bad things we already knew about bad software. I have always enjoyed the anecdote that in China, Tencent had over 6k mediocre engineers servicing QQ then hired fewer than 30 great ones to build the core of WeChat... AI isn't exactly free and software maintenance doesn't scale linearly
  - Aeolun 1y ago
    > But slop piled on top of slop piled on top of slop is only going to compound all the bad things we already knew about bad software While that is true, AI isn’t going to make the big difference here. Whether the slop is written by AI or 6000 mediocre engineers is of no matter to the end result. One might argue that if it were written by AI at least those engineers could do something useful with their lives.
torben-friis 1y ago
As the op, for the case I was thinking about, it’s “couldn’t” as in “I don’t have the time to go checking file by file and the variation is not straightforward enough that grepping will surface cases straightforwardly”. I’m very much able to understand the result and test for consequences, I wouldn’t think of putting code I don’t understand in production.
johnisgood 1y ago
> Your comment makes it sound like you're now dependent on AI to refactor again Not necessarily. It may have refactored the codebase in a way that is more organized and easier to follow. > how did you guarantee that the change offered by the AI made proper sense and didn't leave out critical patterns that were too complex for you to detect ? Perhaps extensive testing? Or a prayer.
throwaway2037 1y ago
Before I started using advanced IDEs that could navigate project structures very quickly, it was normal to have a relatively poor visibility -- call it "fog of war/code". In a 500,000 line C++ project (I have seen a few in my career), as a junior dev, I might only understand a few thousand lines from a few files I have studied. And, I had very little idea of the overall architecture. I see LLMs here as a big opportunity. I assume that most huge software projects developed by non-tech companies look pretty similar -- organic, and poorly documented and tested.
- markus_zhang 1y ago
  What do you think about software such as source insight that gives developers an eagle eye of view of the project?
  - throwaway2037 1y ago
    You raise a good point. I had a former teammate who swore by Source Insight. To repeat myself, I wrote: <<Before I started using advanced IDEs that could navigate project structures very quickly>>. So, I was really talking about my life before I started using advanced IDEs. It was so hard to get a good grasp of a project and navigate quickly.
    - markus_zhang 1y ago
      This makes sense.
- guappa 1y ago
  That's what documentation is for. If you don't have that, AI won't figure it out either.
  - jrochkind1 1y ago
    I'm not sure that's true?

firecall 1y ago

That's an important distinction! There's a difference between not intellectually understanding something and not being able to refactor something because if you start pulling on a thread, you are not sure what will unravel! And often there just isn't time allocated in a budget to begin an unlimited game of bug testing whack-a-mole!

OccamsMirror 1y ago
To makeitdouble's point, how is this any different with an LLM provided solution? What confidence do you have that isn't also beginning an unlimited game of bug testing whack-a-mole? My confidence in LLMs is not that high and I use Claude a lot. The limitations are very apparent very quickly. They're great for simple refactors and doing some busy work, but if you're refactoring something you're too afraid to do by hand then I fear you've simply deferred responsibility to the LLM - assuming it will understand the code better than you do, which seems foolhardy.
- rocqua 1y ago
  Especially with refactoring, it tends to be tedious and repetitive work that is slightly too complicated for a (regex) search replace. A lot of repetitive slight variations on the same easy to describe change sounds pretty good to ask an LLM to do quickly.

pshc 1y ago

Sometimes a problem is a weird combination of hairy/obscure/tedious where I simply don’t have the activation energy to get started. Like, I could do it with a gun to my head. But if someone else were to do it for me I would gratefully review the merge request.

strogonoff 1y ago
Reviewing a merge request should require at least the same activation energy as writing the solution yourself, as in order to adequately evaluate a solution you first need to acquire a reference point in mind as to what the right solution should be in the first place. For me personally, the activation energy is higher when reviewing: it’s fun to come up with the solution that ends up being used, not so fun to come up with a solution that just serves as a reference point for evaluation and then gets immediately thrown away. Plus, I know in advance that a lot of cycles will be wasted on trying to understand how someone else’s vision maps onto my solution, especially when that vision is muddy.
- solaire_oa 1y ago
  The submitter should also have thoroughly reviewed their own MR/PR. Even before LLMs, coders not having reviewed their own code would be completely discourteous and disrespectful to the reviewer. It's an embarrassing faux pas that makes the submitter and the team all look and feel bad when there are obvious problems that need to be called out and fixed. Submitting LLM barf for review and not reviewing it should be grounds for termination. The only way I can envision LLM barf being sustainable, or plausible, is if you removed code review altogether.
  - strogonoff 1y ago
    > The submitter should also have thoroughly reviewed their own MR/PR What does it mean to have to review your own code as a separate activity? Do many people contribute code that they wrote but… never read? > Submitting LLM barf Oh right…
    - 1y ago

Aeolun 1y ago

I think, if it’s similar to how I feel about it, that it’s more about always being able to do it, but not wanting to expend the mental effort to correctly adjust all those 30 places. Your boss is not going to care, so while it’s a bit better going forward, justifying the time to do it manually doesn’t make sense even to yourself. If you can do it using an LLM in a few hours however, suddenly making your life, and the lives of everyone that comes after you, easier becomes a pretty simple decision.

djtango 1y ago
So everyone is talking across each other... AI is a sharp tool, use it well and it cuts. Use it poorly and it'll cut you. Helping you overcome the activation barrier to make that redactor is great if that truly is what it is. That is probably still worth billions in the aggregate given git is considered billion dollar software. But slop piled on top of slop piled on top of slop is only going to compound all the bad things we already knew about bad software. I have always enjoyed the anecdote that in China, Tencent had over 6k mediocre engineers servicing QQ then hired fewer than 30 great ones to build the core of WeChat... AI isn't exactly free and software maintenance doesn't scale linearly
- Aeolun 1y ago
  > But slop piled on top of slop piled on top of slop is only going to compound all the bad things we already knew about bad software While that is true, AI isn’t going to make the big difference here. Whether the slop is written by AI or 6000 mediocre engineers is of no matter to the end result. One might argue that if it were written by AI at least those engineers could do something useful with their lives.

torben-friis 1y ago

As the op, for the case I was thinking about, it’s “couldn’t” as in “I don’t have the time to go checking file by file and the variation is not straightforward enough that grepping will surface cases straightforwardly”. I’m very much able to understand the result and test for consequences, I wouldn’t think of putting code I don’t understand in production.

johnisgood 1y ago

> Your comment makes it sound like you're now dependent on AI to refactor again Not necessarily. It may have refactored the codebase in a way that is more organized and easier to follow. > how did you guarantee that the change offered by the AI made proper sense and didn't leave out critical patterns that were too complex for you to detect ? Perhaps extensive testing? Or a prayer.

throwaway2037 1y ago

Before I started using advanced IDEs that could navigate project structures very quickly, it was normal to have a relatively poor visibility -- call it "fog of war/code". In a 500,000 line C++ project (I have seen a few in my career), as a junior dev, I might only understand a few thousand lines from a few files I have studied. And, I had very little idea of the overall architecture. I see LLMs here as a big opportunity. I assume that most huge software projects developed by non-tech companies look pretty similar -- organic, and poorly documented and tested.

markus_zhang 1y ago
What do you think about software such as source insight that gives developers an eagle eye of view of the project?
- throwaway2037 1y ago
  You raise a good point. I had a former teammate who swore by Source Insight. To repeat myself, I wrote: <<Before I started using advanced IDEs that could navigate project structures very quickly>>. So, I was really talking about my life before I started using advanced IDEs. It was so hard to get a good grasp of a project and navigate quickly.
  - markus_zhang 1y ago
    This makes sense.
guappa 1y ago
That's what documentation is for. If you don't have that, AI won't figure it out either.
- jrochkind1 1y ago
  I'm not sure that's true?