3 ms·

It's... really the same problem when you hire people to just write tests. A lot of time it just confirms that the code does what the code does. Having clear spe

It's... really the same problem when you hire people to just write tests. A lot of time it just confirms that the code does what the code does. Having clear specs of what the code should do make things better and clearer.

aray07 7mo ago
yup agree - i think have specs and then do verifications against the spec. I have heard that this is how a lot of consulting firms work - you have acceptance criterias and thats how work is validated.
SoftTalker 7mo ago
Yep, tests written after the fact are just verifying tautologies. > Most teams don't [write tests first] because thinking through what the code should do before writing it takes time they don't have. It's astonishing to me how much our industry repeats the same mistakes over and over. This doesn't seem like what other engineering disciplines do. Or is this just me not knowing what it looks like behind the curtain of those fields?
- yurishimo 7mo ago
  When push comes to shove, software can usually be fudged. Unlike a building or a water treatment plant where the first fuck up could mean that people die. I like to think that people writing actual mission critical software try their absolute best to get it right before shipping and that the rest our industry exists in a totally separate world where a bug in the code is just actually not that big of a deal. Yeah, it might be expensive to fix, but usually it can be reverted or patched with only an inconvenience to the user and to the business. It’s like the fines that multinational companies pay when breaking the law. If it’s a cost of doing business, it’s baked into the price of the product. You see this also in other industries. OSHA violations on a residential construction site? I bet you can find a dozen if you really care to look. But 99% of the time, there are no consequences big enough for people to care so nobody wears their PPE because it “slows them down” or “makes them less nimble”. Sound familiar?
  - girvo 7mo ago
    Quite. We’re far more similar to construction workers than we are civil engineers, despite the lofty title we like to bestow upon ourselves.
    - Ekaros 7mo ago
      And from slightly different view. What we make is not output of modern mass production. With highly tuned and most of time perfectly matching parts build into one unit. Instead we make pre-mass production bespoke products where each part is slightly filled and fitted together from bunch of random components. Say the barrel can't be changed between two different handguns. We just have magic technology to replicate the single gun multiple times. Does not mean it is actually mass-produced in sense say our current power tools are.
  - AlotOfReading 7mo ago
    I like to think that people writing actual mission critical software try their absolute best to get it right before shipping. People try, but the only fundamentally different part is that you spend time thinking about and documenting your process rather than just doing it. There's always one more bug. Usually there ends up being a human covering up for the system's failures somewhere that no one else notices. That's the driver in the car, or the factory tech who adjusts things just a bit.
- gonzalohm 7mo ago
  That's probably dependent on your specific area of work. For most projects, It's okayish to deploy code with bugs. There will be future releases that fix those bugs and add improvements. Obviously that's not the case with high risk systems like space rockets software and similar. With other engineering professions, all projects are like that. You cannot "deploy a bridge to production" to see what happens and fix it after a few have died
- hrmtst93837 7mo ago
  [flagged]
tibbar 7mo ago
a lot of the value of tests is confirming that the system hasn't regressed beyond the behavior at the original release. It's bad if the original release is wrong, but a separate issue is if the system later accidentally stops behaving the way it did originally.
- InsideOutSanta 7mo ago
  The issue I see is that the high test coverage created by having LLMs write tests results in almost all non-trivial changes breaking tests, even if they don't change behavior in ways that are visible from the outside. In one project I work, we require 100% test coverage, so people just have LLMs write tons of tests, and now every change I make to the code base always breaks tests. So now people just ignore broken tests. > Claude, please implement this feature. > Claude, please fix the tests. The only thing we've gained from this is that we can brag about test coverage.
  - mattmanser 7mo ago
    I feel it end up a massive drag on development velocity and makes refactoring to simpler designs incredibly painful. But hey, we're just supposed to let the AIs run wild and rewrite everything every change so maybe that's a heretic view.
    - duskdozer 7mo ago
      >simpler designs Some complex design might just be hacks on hacks, but some are chesterton's fences
  - ForHackernews 7mo ago
    Unit tests vs acceptance tests. You shouldn't be afraid to throw away unit tests if the implementation changes, and acceptance tests should verify behavior at API boundaries, ignoring implementation details.
    - hinkley 7mo ago
deleted 7mo ago
[deleted]
predkambrij 7mo ago
Yeah. The purpose of tests is to verify correctness of their setUp mocks :)

aray07 7mo ago

yup agree - i think have specs and then do verifications against the spec. I have heard that this is how a lot of consulting firms work - you have acceptance criterias and thats how work is validated.

SoftTalker 7mo ago

Yep, tests written after the fact are just verifying tautologies. > Most teams don't [write tests first] because thinking through what the code should do before writing it takes time they don't have. It's astonishing to me how much our industry repeats the same mistakes over and over. This doesn't seem like what other engineering disciplines do. Or is this just me not knowing what it looks like behind the curtain of those fields?

yurishimo 7mo ago
When push comes to shove, software can usually be fudged. Unlike a building or a water treatment plant where the first fuck up could mean that people die. I like to think that people writing actual mission critical software try their absolute best to get it right before shipping and that the rest our industry exists in a totally separate world where a bug in the code is just actually not that big of a deal. Yeah, it might be expensive to fix, but usually it can be reverted or patched with only an inconvenience to the user and to the business. It’s like the fines that multinational companies pay when breaking the law. If it’s a cost of doing business, it’s baked into the price of the product. You see this also in other industries. OSHA violations on a residential construction site? I bet you can find a dozen if you really care to look. But 99% of the time, there are no consequences big enough for people to care so nobody wears their PPE because it “slows them down” or “makes them less nimble”. Sound familiar?
- girvo 7mo ago
  Quite. We’re far more similar to construction workers than we are civil engineers, despite the lofty title we like to bestow upon ourselves.
  - Ekaros 7mo ago
    And from slightly different view. What we make is not output of modern mass production. With highly tuned and most of time perfectly matching parts build into one unit. Instead we make pre-mass production bespoke products where each part is slightly filled and fitted together from bunch of random components. Say the barrel can't be changed between two different handguns. We just have magic technology to replicate the single gun multiple times. Does not mean it is actually mass-produced in sense say our current power tools are.
- AlotOfReading 7mo ago
  I like to think that people writing actual mission critical software try their absolute best to get it right before shipping. People try, but the only fundamentally different part is that you spend time thinking about and documenting your process rather than just doing it. There's always one more bug. Usually there ends up being a human covering up for the system's failures somewhere that no one else notices. That's the driver in the car, or the factory tech who adjusts things just a bit.
gonzalohm 7mo ago
That's probably dependent on your specific area of work. For most projects, It's okayish to deploy code with bugs. There will be future releases that fix those bugs and add improvements. Obviously that's not the case with high risk systems like space rockets software and similar. With other engineering professions, all projects are like that. You cannot "deploy a bridge to production" to see what happens and fix it after a few have died
hrmtst93837 7mo ago
[flagged]

tibbar 7mo ago

a lot of the value of tests is confirming that the system hasn't regressed beyond the behavior at the original release. It's bad if the original release is wrong, but a separate issue is if the system later accidentally stops behaving the way it did originally.

InsideOutSanta 7mo ago
The issue I see is that the high test coverage created by having LLMs write tests results in almost all non-trivial changes breaking tests, even if they don't change behavior in ways that are visible from the outside. In one project I work, we require 100% test coverage, so people just have LLMs write tons of tests, and now every change I make to the code base always breaks tests. So now people just ignore broken tests. > Claude, please implement this feature. > Claude, please fix the tests. The only thing we've gained from this is that we can brag about test coverage.
- mattmanser 7mo ago
  I feel it end up a massive drag on development velocity and makes refactoring to simpler designs incredibly painful. But hey, we're just supposed to let the AIs run wild and rewrite everything every change so maybe that's a heretic view.
  - duskdozer 7mo ago
    >simpler designs Some complex design might just be hacks on hacks, but some are chesterton's fences
- ForHackernews 7mo ago
  Unit tests vs acceptance tests. You shouldn't be afraid to throw away unit tests if the implementation changes, and acceptance tests should verify behavior at API boundaries, ignoring implementation details.
  - hinkley 7mo ago

deleted 7mo ago

[deleted]

predkambrij 7mo ago

Yeah. The purpose of tests is to verify correctness of their setUp mocks :)