3 ms·

Agreed. Good tests are an order of magnitude harder than good code.

postalrat 3y ago
Which is why they should be treated as a waste of time unless specific tests can be justified.
brabel 3y ago
I don't know where you're getting that from but it's simply wrong. Testing is quite easy IMO. I've been working at the same place for almost 10 years and I introduced our testing framework. We have hundreds of thousands of tests. New guys have some trouble to get started, but after a couple of months they're writing tests for our systems like a pro. Most tests are use-case based or written for checking error-handling. Use-case tests are easy to write: you don't even need to be a programmer (in fact, it's good if use-cases are defined by a Product Owner or Tester), though of course some cases are only known to the programmer as they're the only ones who dive into the details. The programmer should come up with all use-cases the PO missed, of course, and judge whether or not they need to test those too... sometimes it's ok to not test as the cost-benefit is low. Anyway, once you have this use-case based test mentality, it's very easy to write the tests (using a proper language to do it is important! Don't use just JUnit if you're doing Java as it will be really tedious to write and you will stop midway - I know, I've been there... I highly recommend using Spock, though other frameworks to make writing test pleasurable exist). This applies mostly for "integration tests". For unit tests, hopefully you don't find them difficult to write?! I find them quite easy to write since I know how to write testable code, which takes a while to learn but once you do, it's really easy. If you have examples of difficult to write tests, I would be curious to see it! Perhaps we can discuss how to make them easy.
- viraptor 3y ago
  The positive ones (use cases) are usually pretty straightforward. It's once you get to failures like "how does the whole system recover if one packet, 3 steps into a transaction is corrupted". If the system is complex enough, that warrants a whole internal fault injection framework and can take a really long time to make it reliable and usable. To be fair most projects don't care beyond "rollback the database and maybe display a 5xx error or something". But some do. Anyway, use cases are fine. It's the failures / edge cases that cause pain.
- IshKebab 3y ago
  You can't just say "testing is quite easy". It totally depends on what you're testing and how thoroughly you want to test it. For example GUI testing is IMO still an unsolved problem. Maybe AI will help there but existing solutions are generally not worth the pain. I work in silicon verification and the testing we do is way way way more thorough than software testing, for obvious reasons. Do you formally verify your software? Unlikely. I can only assume you work in an easy-to-test domain on a project that doesn't have changing requirements, like... I dunno a C compiler or something.
  - brabel 3y ago
    The kind of application we test is probably more complex than most as we're a product company supporting a huge number of integrations and specifications. You're dismissing my claim by basically saying I am naive. Which is not an honest argument as you know absolutely nothing about me (and I don't want to tell you more than I did here). About changing requirements: what does that have to do with testing at all? If requirements change, you basically discard the tests for the old behaviour and start over... I would say that the testing we do is very close to formal verification because it's close to being comprehensive - though no, we do not use methods normally classified as such. I tried to but the benefit we would get over our current approach would be negligible. By the way, I do a lot of UI testing, and dare I say it: yes, it's easy too. We use this sort of thing if you're curious: https://gebish.org/manual/current/#pages https://gebish.org/manual/current/#pages Again, if you find a real example of something you find hard to test, let me know so I can evaluate it against my own situation.

Which is why they should be treated as a waste of time unless specific tests can be justified.

I don't know where you're getting that from but it's simply wrong. Testing is quite easy IMO. I've been working at the same place for almost 10 years and I introduced our testing framework. We have hundreds of thousands of tests. New guys have some trouble to get started, but after a couple of months they're writing tests for our systems like a pro. Most tests are use-case based or written for checking error-handling. Use-case tests are easy to write: you don't even need to be a programmer (in fact, it's good if use-cases are defined by a Product Owner or Tester), though of course some cases are only known to the programmer as they're the only ones who dive into the details. The programmer should come up with all use-cases the PO missed, of course, and judge whether or not they need to test those too... sometimes it's ok to not test as the cost-benefit is low. Anyway, once you have this use-case based test mentality, it's very easy to write the tests (using a proper language to do it is important! Don't use just JUnit if you're doing Java as it will be really tedious to write and you will stop midway - I know, I've been there... I highly recommend using Spock, though other frameworks to make writing test pleasurable exist). This applies mostly for "integration tests". For unit tests, hopefully you don't find them difficult to write?! I find them quite easy to write since I know how to write testable code, which takes a while to learn but once you do, it's really easy. If you have examples of difficult to write tests, I would be curious to see it! Perhaps we can discuss how to make them easy.

viraptor 3y ago
The positive ones (use cases) are usually pretty straightforward. It's once you get to failures like "how does the whole system recover if one packet, 3 steps into a transaction is corrupted". If the system is complex enough, that warrants a whole internal fault injection framework and can take a really long time to make it reliable and usable. To be fair most projects don't care beyond "rollback the database and maybe display a 5xx error or something". But some do. Anyway, use cases are fine. It's the failures / edge cases that cause pain.
IshKebab 3y ago
You can't just say "testing is quite easy". It totally depends on what you're testing and how thoroughly you want to test it. For example GUI testing is IMO still an unsolved problem. Maybe AI will help there but existing solutions are generally not worth the pain. I work in silicon verification and the testing we do is way way way more thorough than software testing, for obvious reasons. Do you formally verify your software? Unlikely. I can only assume you work in an easy-to-test domain on a project that doesn't have changing requirements, like... I dunno a C compiler or something.
- brabel 3y ago
  The kind of application we test is probably more complex than most as we're a product company supporting a huge number of integrations and specifications. You're dismissing my claim by basically saying I am naive. Which is not an honest argument as you know absolutely nothing about me (and I don't want to tell you more than I did here). About changing requirements: what does that have to do with testing at all? If requirements change, you basically discard the tests for the old behaviour and start over... I would say that the testing we do is very close to formal verification because it's close to being comprehensive - though no, we do not use methods normally classified as such. I tried to but the benefit we would get over our current approach would be negligible. By the way, I do a lot of UI testing, and dare I say it: yes, it's easy too. We use this sort of thing if you're curious: https://gebish.org/manual/current/#pages https://gebish.org/manual/current/#pages Again, if you find a real example of something you find hard to test, let me know so I can evaluate it against my own situation.