4 ms·

Eventual convergence? Assuming each defect fix has a 30% chance of introducing a new defect, we keep cycling until done?

Kinrany 7mo ago
Why would it converge?
saintfire 7mo ago
Assuming you can catch every new bug it introduces. Both assumptions being unlikely. You also end up with a code base you let an AI agent trample until it is satisfied; ballooned in complexity and redudant brittle code.
- charcircuit 7mo ago
  You can have an AI agent refactor and improve code quality.
  - abakker 7mo ago
    But, have you any code that has been vetted and verified to see if this approach works? This whole Agentic code quality claim is an assertion, but where is the literal proof?
    - WithinReason 7mo ago
      If it can be trained with reinforcement learning then it will happen
    - wredcoll 7mo ago
      Did we have code quality before llms?
  - intended 7mo ago
    It’s agents all the way down - until you have liability. At some point, it’s going to be someone’s neck on the line, and saying “the agents know” isn’t going to satisfy customers (or in a worst case, courts).
    - charcircuit 7mo ago
      Sure it can. It's not like humans aren't already deflecting liability or moving it to insurance agencies.
      - lelanthran 7mo ago
        > It's not like humans aren't already deflecting liability They attempt to, sure, but it rarely works. Now, with AI, maybe it might, but that's sort of a worse outcome for the specific human involved - "If you're just an intermediary between the AI and me, WTF do I need you for?" > or moving it to insurance agencies. They aren't "moving" it to insurance companies, they are amortising the cost of the liability at a small extra cost. That's a big difference.
      - intended 7mo ago
        At some point, the risk/return calculus becomes too expensive for insurance companies. Usually thats after the premiums become too high for most people to pay.
    - wredcoll 7mo ago
      > until you have liability And are you thinking this going to start happening at some point or what? The letters I get every other month telling me I now have free credit monitoring because of a personal info breach seems to suggest otherwise.
      - intended 7mo ago
        A firm has very different amounts of time, ability and money to spend on following up on broken contracts.
  - lmm 7mo ago
    Funnily enough I've literally never seen anyone demo this, despite all the other AI hype. It's the one thing that convinces me they're still behind.
  - flir 7mo ago
    Just today I had an agent add a fourth "special case" to a codebase, and I went back and DRY'd three of them. Now I used the agent to do a lot of the grunt work in that refactor, but it was still a design decision initiated by me. The chatbot, left unattended, would not have seen that needed to be done. (And when, during my refactor, it tried to fold in the fourth case I had to stop it.) (And for a lot of code, that's ok - my static site generator is an unholy mess at this point, and I don't much care. But for paid work...)
majewsky 7mo ago
That's assuming that each fix can only introduce at most one additional defect, which is obviously untrue.
bluefirebrand 7mo ago
As long as we're inventing numbers, what if it's a 90% chance? What if it's a 200% chance, and every fix introduces multiple defects?
pron 7mo ago
Except they don't converge. You see that if you use agents to evolve a codebase. We also saw exactly that in the failed Anthropic experiment to create a C compiler.
nostrademons 7mo ago
The chance of a defect fix introducing a new defect tends to grow linearly with the size of the codebase, since defects are usually caused by the interaction between code and there's now more code to interact with. If you plot this out, you'll notice that it eventually reaches > 100% and the total number of defects will eventually grow exponentially, as each bugfix eventually introduces more bugs than it fixes. Which is what I've actually observed in 25 years in the software industry. The speed at which new bugs are introduced faster than bugfixes varies by organization and the skill of your software architects - good engineers know how to keep coupling down and limit the space of existing code that a new fix could possibly break. I've seen some startups where they reach this asymptote before bringing the product to market though (needless to say, they failed), and it's pretty common for computer games to become steaming piles of shit close to launch, and I've even seen some Google systems killed and rewritten because it became impossible to make forward progress on them. I call this technical bankruptcy, the end result of technical debt.

Kinrany 7mo ago

Why would it converge?

saintfire 7mo ago

Assuming you can catch every new bug it introduces. Both assumptions being unlikely. You also end up with a code base you let an AI agent trample until it is satisfied; ballooned in complexity and redudant brittle code.

charcircuit 7mo ago
You can have an AI agent refactor and improve code quality.
- abakker 7mo ago
  But, have you any code that has been vetted and verified to see if this approach works? This whole Agentic code quality claim is an assertion, but where is the literal proof?
  - WithinReason 7mo ago
    If it can be trained with reinforcement learning then it will happen
  - wredcoll 7mo ago
    Did we have code quality before llms?
- intended 7mo ago
  It’s agents all the way down - until you have liability. At some point, it’s going to be someone’s neck on the line, and saying “the agents know” isn’t going to satisfy customers (or in a worst case, courts).
  - charcircuit 7mo ago
    Sure it can. It's not like humans aren't already deflecting liability or moving it to insurance agencies.
    - lelanthran 7mo ago
      > It's not like humans aren't already deflecting liability They attempt to, sure, but it rarely works. Now, with AI, maybe it might, but that's sort of a worse outcome for the specific human involved - "If you're just an intermediary between the AI and me, WTF do I need you for?" > or moving it to insurance agencies. They aren't "moving" it to insurance companies, they are amortising the cost of the liability at a small extra cost. That's a big difference.
    - intended 7mo ago
      At some point, the risk/return calculus becomes too expensive for insurance companies. Usually thats after the premiums become too high for most people to pay.
  - wredcoll 7mo ago
    > until you have liability And are you thinking this going to start happening at some point or what? The letters I get every other month telling me I now have free credit monitoring because of a personal info breach seems to suggest otherwise.
    - intended 7mo ago
      A firm has very different amounts of time, ability and money to spend on following up on broken contracts.
- lmm 7mo ago
  Funnily enough I've literally never seen anyone demo this, despite all the other AI hype. It's the one thing that convinces me they're still behind.
- flir 7mo ago
  Just today I had an agent add a fourth "special case" to a codebase, and I went back and DRY'd three of them. Now I used the agent to do a lot of the grunt work in that refactor, but it was still a design decision initiated by me. The chatbot, left unattended, would not have seen that needed to be done. (And when, during my refactor, it tried to fold in the fourth case I had to stop it.) (And for a lot of code, that's ok - my static site generator is an unholy mess at this point, and I don't much care. But for paid work...)

majewsky 7mo ago

That's assuming that each fix can only introduce at most one additional defect, which is obviously untrue.

bluefirebrand 7mo ago

As long as we're inventing numbers, what if it's a 90% chance? What if it's a 200% chance, and every fix introduces multiple defects?

pron 7mo ago

Except they don't converge. You see that if you use agents to evolve a codebase. We also saw exactly that in the failed Anthropic experiment to create a C compiler.

nostrademons 7mo ago

The chance of a defect fix introducing a new defect tends to grow linearly with the size of the codebase, since defects are usually caused by the interaction between code and there's now more code to interact with. If you plot this out, you'll notice that it eventually reaches > 100% and the total number of defects will eventually grow exponentially, as each bugfix eventually introduces more bugs than it fixes. Which is what I've actually observed in 25 years in the software industry. The speed at which new bugs are introduced faster than bugfixes varies by organization and the skill of your software architects - good engineers know how to keep coupling down and limit the space of existing code that a new fix could possibly break. I've seen some startups where they reach this asymptote before bringing the product to market though (needless to say, they failed), and it's pretty common for computer games to become steaming piles of shit close to launch, and I've even seen some Google systems killed and rewritten because it became impossible to make forward progress on them. I call this technical bankruptcy, the end result of technical debt.