5 ms·

It used to be that you need a good reason to make huge refactorings, because it’s often so much work. Now agent can rewrite half of your code if your prompt is

It used to be that you need a good reason to make huge refactorings, because it’s often so much work. Now agent can rewrite half of your code if your prompt is vague enough and you don’t actual try to review it all. And so the “soul” of a program can change dramatically every single day. It’s both great and very much not so.

Xirdus 3mo ago
The biggest obstacle to huge refactoring has always been minimizing the risk of bugs, not losing any features, and ensuring compatibility with the existing ecosystem. The reason it's become easier in the age of AI is because we stopped caring about these things.
- sarchertech 3mo ago
  Yep. That’s what people are forgetting. If you have an application that many people depend on to do real work, to make money, you won’t survive if you allow AI to constantly make huge changes. Your test suite doesn’t cover all workflows. It doesn’t cover every combination of actions a user can take. So every big AI refactor while change some of those. If this is happening frequently, your software will feel like a janky piece of unusable crap.
- theshrike79 3mo ago
  Actually now we care even more. The reason people didn't write extensive test suites was was because it's super fucking boring to do so. AIs don't care, they'll happily write 50 unit tests with slight variations and pair them with a full dockerized end to end test suite. Now we have at least SOME tests. Are they good? Maybe, maybe not - but we have them. If one of them fails later on, we can check if it's an actual issue or the test being bad. Which is infinitely better than having no tests because nobody can be arsed to write them or the customer isn't paying for the extra hours needed for a proper test suite.
  - zahlman 3mo ago
    Having slop tests instead of no tests is definitely not the same thing as actually caring about avoiding bugs.
    - theshrike79 3mo ago
      This is not about caring. This is about the real world where actual human programmers usually don't want to spend any extra time writing test suites or management doesn't give them the time to do so. So if the starting point is "no tests", having a bunch of "slop tests" is better, right?
  - Xirdus 3mo ago
    People didn't write tests not because it was boring, but because they didn't have time to do it because implementation is more important. That problem didn't go away - it's just instead of not having time to do tests, now we don't have time to review whether the AI wrote tests that make sense. And most of the time it doesn't. They're running the code but not asserting anything. Or they're asserting test inputs instead of implementation outputs. Or outright swallowing errors to make tests pass. Having such tests is, in fact, worse than not having tests at all. Non-existent tests are just as useful as bad tests, and are much cheaper to maintain.
deleted 3mo ago
[deleted]
Cthulhu_ 3mo ago
If the "soul" of a program (which is a vague term but I think I get what you mean) changes daily, that's indicative of a lot of churn, and a lot of churn is not a good idea in any project. it's like changing how the tower of babel should be built daily. Just because you can doesn't mean you should.
- theshrike79 3mo ago
  But if all the tests pass, does it matter what's inside?
  - timothycoleman 3mo ago
    Unfortunately the tests can't (usually) cover all behaviours. And perhaps more importantly, they don't capture more abstract properties of the code like maintainability. AI works best in well maintained code, but unless care is taken, the AI will (today, anyway) make the code less well maintained as it goes. If the AI is allowed to introduce mess into the codebase faster than its ability to deal with the mess increases, the codebase will eventually run into a problem that it's hard to recover from.
    - theshrike79 3mo ago
      And that's kind of the point, some manual testing is always needed - even when humans write the code. But you can test things and behaviours to a sufficient degree. Nobody really does 100% test coverage on real world code (outside of medical, airplanes and space of course).
  - zahlman 3mo ago
    Yes, of course it does. It's strange to me that this can be asked.
    - theshrike79 3mo ago
      Why? How far do you take this, do you inspect the inside of every single thing and program you use? Your microwave promises to heat your food at 800W for 1 minute, do you open it up and measure the magnetometer strength? Do you double-check the timer that it's accurate? You want a program that takes in A and returns B within X milliseconds, I deliver it to you. Do you spend the time going through the whole program line by line to see how it was made or do you trust it? Or when you buy a car do you take it to a garage and dismantle it to see whether it was correctly assembled in the factory? Or do you trust them?

Xirdus 3mo ago

The biggest obstacle to huge refactoring has always been minimizing the risk of bugs, not losing any features, and ensuring compatibility with the existing ecosystem. The reason it's become easier in the age of AI is because we stopped caring about these things.

sarchertech 3mo ago
Yep. That’s what people are forgetting. If you have an application that many people depend on to do real work, to make money, you won’t survive if you allow AI to constantly make huge changes. Your test suite doesn’t cover all workflows. It doesn’t cover every combination of actions a user can take. So every big AI refactor while change some of those. If this is happening frequently, your software will feel like a janky piece of unusable crap.
theshrike79 3mo ago
Actually now we care even more. The reason people didn't write extensive test suites was was because it's super fucking boring to do so. AIs don't care, they'll happily write 50 unit tests with slight variations and pair them with a full dockerized end to end test suite. Now we have at least SOME tests. Are they good? Maybe, maybe not - but we have them. If one of them fails later on, we can check if it's an actual issue or the test being bad. Which is infinitely better than having no tests because nobody can be arsed to write them or the customer isn't paying for the extra hours needed for a proper test suite.
- zahlman 3mo ago
  Having slop tests instead of no tests is definitely not the same thing as actually caring about avoiding bugs.
  - theshrike79 3mo ago
    This is not about caring. This is about the real world where actual human programmers usually don't want to spend any extra time writing test suites or management doesn't give them the time to do so. So if the starting point is "no tests", having a bunch of "slop tests" is better, right?
- Xirdus 3mo ago
  People didn't write tests not because it was boring, but because they didn't have time to do it because implementation is more important. That problem didn't go away - it's just instead of not having time to do tests, now we don't have time to review whether the AI wrote tests that make sense. And most of the time it doesn't. They're running the code but not asserting anything. Or they're asserting test inputs instead of implementation outputs. Or outright swallowing errors to make tests pass. Having such tests is, in fact, worse than not having tests at all. Non-existent tests are just as useful as bad tests, and are much cheaper to maintain.

deleted 3mo ago

[deleted]

Cthulhu_ 3mo ago

If the "soul" of a program (which is a vague term but I think I get what you mean) changes daily, that's indicative of a lot of churn, and a lot of churn is not a good idea in any project. it's like changing how the tower of babel should be built daily. Just because you can doesn't mean you should.

theshrike79 3mo ago
But if all the tests pass, does it matter what's inside?
- timothycoleman 3mo ago
  Unfortunately the tests can't (usually) cover all behaviours. And perhaps more importantly, they don't capture more abstract properties of the code like maintainability. AI works best in well maintained code, but unless care is taken, the AI will (today, anyway) make the code less well maintained as it goes. If the AI is allowed to introduce mess into the codebase faster than its ability to deal with the mess increases, the codebase will eventually run into a problem that it's hard to recover from.
  - theshrike79 3mo ago
    And that's kind of the point, some manual testing is always needed - even when humans write the code. But you can test things and behaviours to a sufficient degree. Nobody really does 100% test coverage on real world code (outside of medical, airplanes and space of course).
- zahlman 3mo ago
  Yes, of course it does. It's strange to me that this can be asked.
  - theshrike79 3mo ago
    Why? How far do you take this, do you inspect the inside of every single thing and program you use? Your microwave promises to heat your food at 800W for 1 minute, do you open it up and measure the magnetometer strength? Do you double-check the timer that it's accurate? You want a program that takes in A and returns B within X milliseconds, I deliver it to you. Do you spend the time going through the whole program line by line to see how it was made or do you trust it? Or when you buy a car do you take it to a garage and dismantle it to see whether it was correctly assembled in the factory? Or do you trust them?