3 ms·

Unit tests are mostly useless. If one wants their tests to be anything more than a happy path and sad path that the developer thought up (combined with an accr

Unit tests are mostly useless. If one wants their tests to be anything more than a happy path and sad path that the developer thought up (combined with an accretion of sad paths that were found later as bugs), then one would do themselves a tremendous favor by adopting property-based testing and mutation testing. Doing so forces you to define several constraint-based models for your program, and then layering mutation testing over the top helps you figure out how good your tests actually are.

deleted 10y ago
[deleted]
CraigJPerry 10y ago
>> Unit tests are mostly useless The speed difference though, integration tests just take too long to be able to completely replace unit tests.
- brianwawok 10y ago
  It depends. Both on project size and how your tests parallelize. If you can integration test an entire project in 30 seconds, that is fast enough. Fast enough to have it in a loop for by the book TDD? No but I'm not sure by the book TDD adds any value. You mash test once before committing and if all is good you go ahead.
  - sitkack 10y ago
    With a decent hierarchical decomposition, integration testing at component/module/subsystem boundary achieves both good coverage and low latency test results. I worked on a big ETL database migration tool in Python and it had a top level integration test with the smallest possible subset of data, like 500k end to end. Then each subsystem (it was architected like a nanopass compiler) had a more complex set of integration tests that only checked that pass but more rigorously. The whole project (10k lines of code) had under 10 unit tests that double checked tricky low level primitives. Every test was independent and could be run in parallel, my longest test was just under a minute, my shortest was about 1 sec. The shortest tests were run on module import ensuring both an easy affordance to testing and a tight feedback loop with no unit tests to impede refactoring.
sametmax 10y ago
I see a uncredible difference in the architecture quality in projects with unit tests. People test forces people to think about interface, forces them to use their code, torture it, break it, knowing it in and out and see the limits. Just for the intellectual tool UT are, they are valuable. Now they have a cost, and sometimes they are not worth it, but they are certainly not useless.
- im_down_w_otp 10y ago
  Writing property-based tests will incentivize you to think about those things too, and it they will actually exercise your code more strenuously across a wide spectrum of inputs and corner cases. Unit tests seem mostly worthwhile as documentation or tutorial for how to use something or for what something is "supposed to" do. But as validation of completeness, correctness, or coverage of your design and implementation... they fall extremely short and leave a lot to be desired. In either case, if software quality and reliability actually matters to your problem domain, then mutation testing should be considered an absolute prerequisite to any kind of testing method (unit, property-based, integration, etc.) otherwise you have almost no way to measure the quality of your tests.

deleted 10y ago

[deleted]

CraigJPerry 10y ago

>> Unit tests are mostly useless The speed difference though, integration tests just take too long to be able to completely replace unit tests.

brianwawok 10y ago
It depends. Both on project size and how your tests parallelize. If you can integration test an entire project in 30 seconds, that is fast enough. Fast enough to have it in a loop for by the book TDD? No but I'm not sure by the book TDD adds any value. You mash test once before committing and if all is good you go ahead.
- sitkack 10y ago
  With a decent hierarchical decomposition, integration testing at component/module/subsystem boundary achieves both good coverage and low latency test results. I worked on a big ETL database migration tool in Python and it had a top level integration test with the smallest possible subset of data, like 500k end to end. Then each subsystem (it was architected like a nanopass compiler) had a more complex set of integration tests that only checked that pass but more rigorously. The whole project (10k lines of code) had under 10 unit tests that double checked tricky low level primitives. Every test was independent and could be run in parallel, my longest test was just under a minute, my shortest was about 1 sec. The shortest tests were run on module import ensuring both an easy affordance to testing and a tight feedback loop with no unit tests to impede refactoring.

sametmax 10y ago

I see a uncredible difference in the architecture quality in projects with unit tests. People test forces people to think about interface, forces them to use their code, torture it, break it, knowing it in and out and see the limits. Just for the intellectual tool UT are, they are valuable. Now they have a cost, and sometimes they are not worth it, but they are certainly not useless.

im_down_w_otp 10y ago
Writing property-based tests will incentivize you to think about those things too, and it they will actually exercise your code more strenuously across a wide spectrum of inputs and corner cases. Unit tests seem mostly worthwhile as documentation or tutorial for how to use something or for what something is "supposed to" do. But as validation of completeness, correctness, or coverage of your design and implementation... they fall extremely short and leave a lot to be desired. In either case, if software quality and reliability actually matters to your problem domain, then mutation testing should be considered an absolute prerequisite to any kind of testing method (unit, property-based, integration, etc.) otherwise you have almost no way to measure the quality of your tests.