3 ms·

> Why anyone would prefer manual testing for anything is beyond me, Formal software testing is great, particularly if you're working on a hard problem with cle

> Why anyone would prefer manual testing for anything is beyond me, Formal software testing is great, particularly if you're working on a hard problem with clear goals like building some hypothetical API to do some financial calculations or transactions where an error might cost tens of thousands of dollars or more. But there's some cases where literal manual testing is the best business decision, even if it's not the best software engineering decision. To give one example: have you ever worked in an agency environment? What do clients do when you hand them off any scale of project? Some people within their organization will then go over every inch of the software and see if it works as they expected. They will literally never ask once what your test coverage is. But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Writing tests in this environment is 99% intellectual masturbation.

kazinator 4y ago
> But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Those people are going to not stop at one; so you will be foolish if you don't write a regression test for the behavior they wanted. When they find the next thing they think is broken, and you try to fix it, you could regress your earlier fix for the first item. In reacting to the user reports, you may be breaking other things that are not tested, but at least you pin down the behavior that the users are reporting. UI element not working right in some way could be a genuine "pass" for testing. I mean, writing some monkey test that feeds events into a UI to check that it's in exactly some expected state could be a big waste of time. Probably if you were writing a reusable widget framework, you might want that sort of testing, because you could make an inadvertant change which makes every instance of some widget behave differently in downstream applications. If you have some complex behavior in your UI that doesn't come from the underlying widgets, and has to be right in certain ways, then that could be worth testing.
Jtsummers 4y ago
Your comment was dead when I saw it so I vouched for it. > But there's some cases where literal manual testing is the best business decision I will agree, manual testing can be the best business decision. Especially if it's someone else's money and time you're spending on it rather than your own. Or if your system is trivial or short-lived. I don't work on short-lived systems, and manual testing has repeatedly been a major hurdle to improvement overall when dealing with non-trivial, long-lived systems in my experience. Manual testing either misses too much, or takes too much time if it's comprehensive. And even if it is comprehensive, or perhaps especially if it's comprehensive, manual testing gives you many false positives and negatives because it is error prone.
- logicalmonster 4y ago
  There's many different worlds out there. I think most of the developers on HN have predominately worked on long-lived products where some culture of heavily testing things makes perfect sense and the idea of doing anything else seems strange. If you've got a customer base on some SaaS that depends on things working as expected, then yeah, having lots of tests to make sure that your system is consistent and changes on various edge cases spread over the course of years don't break subtle things is vital. So there's probably sort of a developer cultural disconnect when somebody who has often built urgently needed, shorter-life, bespoke projects points out that there's some types of projects and environments where software testing brings a much different value-proposition. In some cases, you're not building something that's intended to last for years: maybe you're building a specific software that's intended to be used in a booth demonstration for 3 days in one industry conference, as an example. In some cases, the speed of development and an absolute lowest cost takes priority over perfect software engineering principles, or otherwise something couldn't even be built.
MetaWhirledPeas 4y ago
When Barry from HR doesn't like the way a page works, that's a feature request, or maybe it's a bug. When a bit of errant code refunds the customer $10000 instead of $100, that's a BUG (ALL CAPS). The two are not mutually exclusive. You need human critics looking at what you create, but you also need objective tests, and automation is very good at objective tests. You might even ask Barry to look at your creation earlier, further reducing the need to inject more humans than you already have.

kazinator 4y ago

> But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Those people are going to not stop at one; so you will be foolish if you don't write a regression test for the behavior they wanted. When they find the next thing they think is broken, and you try to fix it, you could regress your earlier fix for the first item. In reacting to the user reports, you may be breaking other things that are not tested, but at least you pin down the behavior that the users are reporting. UI element not working right in some way could be a genuine "pass" for testing. I mean, writing some monkey test that feeds events into a UI to check that it's in exactly some expected state could be a big waste of time. Probably if you were writing a reusable widget framework, you might want that sort of testing, because you could make an inadvertant change which makes every instance of some widget behave differently in downstream applications. If you have some complex behavior in your UI that doesn't come from the underlying widgets, and has to be right in certain ways, then that could be worth testing.

Jtsummers 4y ago

Your comment was dead when I saw it so I vouched for it. > But there's some cases where literal manual testing is the best business decision I will agree, manual testing can be the best business decision. Especially if it's someone else's money and time you're spending on it rather than your own. Or if your system is trivial or short-lived. I don't work on short-lived systems, and manual testing has repeatedly been a major hurdle to improvement overall when dealing with non-trivial, long-lived systems in my experience. Manual testing either misses too much, or takes too much time if it's comprehensive. And even if it is comprehensive, or perhaps especially if it's comprehensive, manual testing gives you many false positives and negatives because it is error prone.

logicalmonster 4y ago
There's many different worlds out there. I think most of the developers on HN have predominately worked on long-lived products where some culture of heavily testing things makes perfect sense and the idea of doing anything else seems strange. If you've got a customer base on some SaaS that depends on things working as expected, then yeah, having lots of tests to make sure that your system is consistent and changes on various edge cases spread over the course of years don't break subtle things is vital. So there's probably sort of a developer cultural disconnect when somebody who has often built urgently needed, shorter-life, bespoke projects points out that there's some types of projects and environments where software testing brings a much different value-proposition. In some cases, you're not building something that's intended to last for years: maybe you're building a specific software that's intended to be used in a booth demonstration for 3 days in one industry conference, as an example. In some cases, the speed of development and an absolute lowest cost takes priority over perfect software engineering principles, or otherwise something couldn't even be built.

MetaWhirledPeas 4y ago

When Barry from HR doesn't like the way a page works, that's a feature request, or maybe it's a bug. When a bit of errant code refunds the customer $10000 instead of $100, that's a BUG (ALL CAPS). The two are not mutually exclusive. You need human critics looking at what you create, but you also need objective tests, and automation is very good at objective tests. You might even ask Barry to look at your creation earlier, further reducing the need to inject more humans than you already have.