3 ms·

Okay, but what do these tests mean? It's easy to add tests that are meaningless and either test the obvious or are just for coverage. But some of the buggiest

Okay, but what do these tests mean? It's easy to add tests that are meaningless and either test the obvious or are just for coverage. But some of the buggiest stuff I've dealt with were in codebases that had full coverage. Because none of the tests were designed to test the original intent of the designed code.

gronky_ 2y ago
I think these tests are useful as regression tests - unit tests can be really helpful when making changes down the line, tipping you off that you missed something. Also much easier to refactor when there’s good test coverage. From the PR: unit tests: what are they good for? Answer: Personal opinion - writing unit testing is not fun. It becomes even less appealing as your codebase grows and maintaining tests becomes a time-consuming chore. However, the benefits of comprehensive unit tests are real: Reliability: They create a more reliable codebase where developers can make changes confidently Speed: Teams can move quickly without fear of breaking existing functionality Safe Refactoring: Code improvements and restructuring become significantly safer when backed by thorough tests Living Documentation: Tests serve as clear documentation of your code's behavior: They show exactly what happens for each input They present changes in a human-readable format: "for this input → expect this output" They run quickly and are easy to execute This immediate feedback loop is beneficial during development
- fzeroracer 2y ago
  > I think these tests are useful as regression tests - unit tests can be really helpful when making changes down the line, tipping you off that you missed something. Also much easier to refactor when there’s good test coverage. The problem is that this assumes that the tests or the method was written correctly in the first place. If the behavior in the method is wrong and the tests are validating that the behavior is wrong, then you pay an extra tax. First to fix the behavior of the method, then to fix the behavior of the tests. That's why automatically generating unit tests is in my opinion adding a bomb to your codebase. The only exception is stuff like basic parameter testing but even that can be questionable at times (is null a valid input at any point for example) unless you know the intent of the code, and AI can't really grasp at the intent.
kevin_nisbet 2y ago
I suspect there is also another angle, which is are the tests maintainable as well? Like you said, if you're not testing intent, this might be one more thing to maintain. In another view, this might just be a fancy way of doing snapshot testing, use AI to generate all the inputs to produce a robust snapshot, but realize the output isn't unit tests, it's snapshots that report changes in outputs that devs will just rubber stamp.

gronky_ 2y ago

I think these tests are useful as regression tests - unit tests can be really helpful when making changes down the line, tipping you off that you missed something. Also much easier to refactor when there’s good test coverage. From the PR: unit tests: what are they good for? Answer: Personal opinion - writing unit testing is not fun. It becomes even less appealing as your codebase grows and maintaining tests becomes a time-consuming chore. However, the benefits of comprehensive unit tests are real: Reliability: They create a more reliable codebase where developers can make changes confidently Speed: Teams can move quickly without fear of breaking existing functionality Safe Refactoring: Code improvements and restructuring become significantly safer when backed by thorough tests Living Documentation: Tests serve as clear documentation of your code's behavior: They show exactly what happens for each input They present changes in a human-readable format: "for this input → expect this output" They run quickly and are easy to execute This immediate feedback loop is beneficial during development

fzeroracer 2y ago
> I think these tests are useful as regression tests - unit tests can be really helpful when making changes down the line, tipping you off that you missed something. Also much easier to refactor when there’s good test coverage. The problem is that this assumes that the tests or the method was written correctly in the first place. If the behavior in the method is wrong and the tests are validating that the behavior is wrong, then you pay an extra tax. First to fix the behavior of the method, then to fix the behavior of the tests. That's why automatically generating unit tests is in my opinion adding a bomb to your codebase. The only exception is stuff like basic parameter testing but even that can be questionable at times (is null a valid input at any point for example) unless you know the intent of the code, and AI can't really grasp at the intent.

kevin_nisbet 2y ago

I suspect there is also another angle, which is are the tests maintainable as well? Like you said, if you're not testing intent, this might be one more thing to maintain. In another view, this might just be a fancy way of doing snapshot testing, use AI to generate all the inputs to produce a robust snapshot, but realize the output isn't unit tests, it's snapshots that report changes in outputs that devs will just rubber stamp.