4 ms·

> During development every functionality is tested each time a minor change is made. Isn’t that much more time consuming than writing a test of any sort? Anyw

> During development every functionality is tested each time a minor change is made. Isn’t that much more time consuming than writing a test of any sort? Anyway that’s why e2e/integration testing with cypress has been becoming one of my favorite kinds of testing. It gives me the security of knowing that a user flow is not broken with a new change. What’s happening under the hood is of little concern and the complex/critical parts also get their unit tests.

Arainach 6y ago
Yes, it is. It also inevitably breaks down. I worked for a long time on a legacy component deep in the guts of Windows. While some new pieces were built in such a way that they could be unit tested, the core framework was a massive piece tightly coupled to itself and several other pieces such that it resisted all attempts to extract interfaces, refactor it into testable chunks, and all the other advice from "Working Effectively With Legacy Code" that I wish I read at the start of my career rather than 7 years into it. The original work was old enough that almost no one was doing unit testing at the time, and as things grew and expanded and depended on it, well...... Anyway, this was the workflow. You made a change, you put it on a machine (or, later, a VM), you ran a test pass. It was a LOT slower than running unit tests, and a lot more error prone because it required the developer to not only know what kinds of tests had to be run, but to actually run them all and not make any mistakes. There were absolutely times where: * A developer new to the area didn't know they had to test something, so they didn't....and their change broke it * A developer made a change and was confident their change couldn't affect an area, so they didn't test it....and their change broke it * A developer thought they were testing something, but had the wrong bits on the machine or missed a step.....and their change broke it * A change was urgently needed and the delta was small, so a developer skipped the testing phase.....and their change broke something Manual testing is painful and not something I ever want to go back to. By the time I left, the general culture around testing at Microsoft was improving, but culture alone can't fix 30-year-old legacy code that was never designed to be tested, so I'm sure there are problem areas like this across the company and industry still. My current team has a much stronger culture around testing - plenty of unit tests, some integration and prod tests, the standard mix - and it's a night and day difference. Even without doing full TDD, the fact that I can write a test case, run it in a few seconds and see it fail, change a few lines of code, run tests in a few seconds and see the test pass makes me orders of magnitude more productive. The fact that I know that the test suite will catch issues prior to checkin (and more tests will run prior to a prod rollout to confirm) makes it much easier, safer, and faster for the entire team to move rapidly and means that I can trust developers who haven't been working in this area to make changes - even if something gets missed in code review, the tests will keep us all honest.
- mgerullis 6y ago
  I feel you. When working on larger applications I regularly ran into this problem: - Adjust code - Adjust unit test - send it to CI - Integration test failed And this was actually a good thing. Despite me being confident about my changes to the code and unit test I realized that another part of the app was failing subsequently. Testing this manually, at least for me, leads to repetitive least-effort testing. Fill the form with whatever garbage it needs to validate and be done. It’s the shortest way to create software that sometimes works but mostly doesn’t. And those people who tested the same workflow upwards of 100 times by hand tend to lose their sanity at some point.
rualca 6y ago
> Isn’t that much more time consuming than writing a test of any sort? I would go a step further and state that it makes absolutely no sense to manually run test batches repeatedly and whenever anyone touches a part of the code. If a task is expected to be automated then automating tests is the only rational decision.

Arainach 6y ago

Yes, it is. It also inevitably breaks down. I worked for a long time on a legacy component deep in the guts of Windows. While some new pieces were built in such a way that they could be unit tested, the core framework was a massive piece tightly coupled to itself and several other pieces such that it resisted all attempts to extract interfaces, refactor it into testable chunks, and all the other advice from "Working Effectively With Legacy Code" that I wish I read at the start of my career rather than 7 years into it. The original work was old enough that almost no one was doing unit testing at the time, and as things grew and expanded and depended on it, well...... Anyway, this was the workflow. You made a change, you put it on a machine (or, later, a VM), you ran a test pass. It was a LOT slower than running unit tests, and a lot more error prone because it required the developer to not only know what kinds of tests had to be run, but to actually run them all and not make any mistakes. There were absolutely times where: * A developer new to the area didn't know they had to test something, so they didn't....and their change broke it * A developer made a change and was confident their change couldn't affect an area, so they didn't test it....and their change broke it * A developer thought they were testing something, but had the wrong bits on the machine or missed a step.....and their change broke it * A change was urgently needed and the delta was small, so a developer skipped the testing phase.....and their change broke something Manual testing is painful and not something I ever want to go back to. By the time I left, the general culture around testing at Microsoft was improving, but culture alone can't fix 30-year-old legacy code that was never designed to be tested, so I'm sure there are problem areas like this across the company and industry still. My current team has a much stronger culture around testing - plenty of unit tests, some integration and prod tests, the standard mix - and it's a night and day difference. Even without doing full TDD, the fact that I can write a test case, run it in a few seconds and see it fail, change a few lines of code, run tests in a few seconds and see the test pass makes me orders of magnitude more productive. The fact that I know that the test suite will catch issues prior to checkin (and more tests will run prior to a prod rollout to confirm) makes it much easier, safer, and faster for the entire team to move rapidly and means that I can trust developers who haven't been working in this area to make changes - even if something gets missed in code review, the tests will keep us all honest.

mgerullis 6y ago
I feel you. When working on larger applications I regularly ran into this problem: - Adjust code - Adjust unit test - send it to CI - Integration test failed And this was actually a good thing. Despite me being confident about my changes to the code and unit test I realized that another part of the app was failing subsequently. Testing this manually, at least for me, leads to repetitive least-effort testing. Fill the form with whatever garbage it needs to validate and be done. It’s the shortest way to create software that sometimes works but mostly doesn’t. And those people who tested the same workflow upwards of 100 times by hand tend to lose their sanity at some point.

rualca 6y ago

> Isn’t that much more time consuming than writing a test of any sort? I would go a step further and state that it makes absolutely no sense to manually run test batches repeatedly and whenever anyone touches a part of the code. If a task is expected to be automated then automating tests is the only rational decision.