3 ms·

People have this aversion of treating code for tests as production code. They spend time designing the architecture for whatever they are implementing but when

People have this aversion of treating code for tests as production code. They spend time designing the architecture for whatever they are implementing but when it comes to writing tests, all of that stuff goes out the window and people just do whatever takes the least amount of time. If you start treating tests as any other code, it should get easier to change as your implementation changes, and you'll get a lot more value out of automated testing.

tetha 3y ago
I've been challenging myself in a current work project. I don't want duplication in tests and I want most methods to have zero setup. The high level tests ideally should just start with a get-request and go straight into expectations. For one, this has resulted in a curious structure. I have a 5 or 6 fixtures with representative sample datasets used in production and these get extended as new features are introduced. If some new addition breaks everything, we'll know immediately, as all tests use the same fixtures and it's all just one place. And it's also instructive during design. These fixtures give you 3-4 situations to think through when adding something new. And the tests are pretty. Most of them have been boiled down into one test_client request and a comparison with a dataset, just 2-3 logical lines. This makes is easy to see if tests are still correct, necessary. Or to just talk about if they make sense overall, even with less technical or involved people.
piva00 3y ago
> If you start treating tests as any other code, it should get easier to change as your implementation changes, and you'll get a lot more value out of automated testing. No, please, don't treat test code as any other production code. Test code can have duplication throughout if it helps readability, test code that relies on abstractions, utility methods, etc. are hard to read, which makes the test cases themselves hard to understand. Test code should be easy to read a case, understand what the test case is preparing for (I like to add comments marking given/when/then sections to make it very clear), what is being called, with what parameters, and what exactly the expected result is. I had my fair share of interactions with test code trying to be smart, test code abstracting away the assertions (into utility methods that themselves call other utilities), test code trying to avoid duplication, and causing headaches and requiring troubleshooting of test code... That is not the reason for test code to exist, it's not to be clean and concise. It's test, you need to know what it is doing, and very quickly assess what failed when a case fails. Please, don't overengineer test code, it should be elegant but there's no need to write it as you would write production code, write it as documentation.
- capableweb 3y ago
  Disagree :) In my experience, test code that has duplication everywhere makes it harder to reliably change the code under test, as there are so many tests that have to be rewritten and you're basically guaranteed to fuck something up. I'm not arguing for making the test cases harder to read, it's very important to be able to see input/output/expected output as always, but it's also important that you're able to change the test code confidently, and duplication actively works against that. I guess the "truth" or "optimal state" is somewhere in-between. Not 100% duplication but also not 0% abstractions.
  - piva00 3y ago
    > I guess the "truth" or "optimal state" is somewhere in-between. Not 100% duplication but also not 0% abstractions. Exactly, I've seen way too much test code that abstracted away a fundamental piece needed for understanding the test setup, which made it much harder to reason about when tests failed. Jumping through the test to the utility used to avoid duplication is another layer of cognitive load, I'm already trying to troubleshoot something, I don't need more things to keep track of while I'm in that process. Rewriting test code is exactly the moment you take to abstract some of it away, when it's been proven that the pattern repeats and is annoying to rewrite. The usual rule of 3 of software engineering. > it's also important that you're able to change the test code confidently, and duplication actively works against that. I disagree with this point a little, if the tests are well written (even if verbose) it should be easy to reconfigure/rewrite them, even with dozens of test cases, it might take a bit more time to rewrite those than just changing a supplier/utility method but it has helped me a lot to actually read the tests. I've been burnt too many times by someone abstracting away the actual call (or a part of it) to the logic under test into an utility to "avoid duplication", that call is the whole point of the test case. One should be able to understand every test case easily so rewriting to accommodate the change made should be straightforward, if it's not then the test case itself is telling there's a code smell, either in the test or in what is under test. The more egregious to me is when tests are a mess of setups not following a simple, straightforward structure. It becomes hard to reason and change over time when the inevitability of growing code happens. Duplication or not won't be the main issue, the structure itself will. I will abstract away some complex setup/adjacent code that isn't under test but anything that the test cases might interact with, and fail because of it I do prefer to be extremely explicit and duplicate code instead of trying to make my tests look tidy and neat.

tetha 3y ago

I've been challenging myself in a current work project. I don't want duplication in tests and I want most methods to have zero setup. The high level tests ideally should just start with a get-request and go straight into expectations. For one, this has resulted in a curious structure. I have a 5 or 6 fixtures with representative sample datasets used in production and these get extended as new features are introduced. If some new addition breaks everything, we'll know immediately, as all tests use the same fixtures and it's all just one place. And it's also instructive during design. These fixtures give you 3-4 situations to think through when adding something new. And the tests are pretty. Most of them have been boiled down into one test_client request and a comparison with a dataset, just 2-3 logical lines. This makes is easy to see if tests are still correct, necessary. Or to just talk about if they make sense overall, even with less technical or involved people.

piva00 3y ago

> If you start treating tests as any other code, it should get easier to change as your implementation changes, and you'll get a lot more value out of automated testing. No, please, don't treat test code as any other production code. Test code can have duplication throughout if it helps readability, test code that relies on abstractions, utility methods, etc. are hard to read, which makes the test cases themselves hard to understand. Test code should be easy to read a case, understand what the test case is preparing for (I like to add comments marking given/when/then sections to make it very clear), what is being called, with what parameters, and what exactly the expected result is. I had my fair share of interactions with test code trying to be smart, test code abstracting away the assertions (into utility methods that themselves call other utilities), test code trying to avoid duplication, and causing headaches and requiring troubleshooting of test code... That is not the reason for test code to exist, it's not to be clean and concise. It's test, you need to know what it is doing, and very quickly assess what failed when a case fails. Please, don't overengineer test code, it should be elegant but there's no need to write it as you would write production code, write it as documentation.

capableweb 3y ago
Disagree :) In my experience, test code that has duplication everywhere makes it harder to reliably change the code under test, as there are so many tests that have to be rewritten and you're basically guaranteed to fuck something up. I'm not arguing for making the test cases harder to read, it's very important to be able to see input/output/expected output as always, but it's also important that you're able to change the test code confidently, and duplication actively works against that. I guess the "truth" or "optimal state" is somewhere in-between. Not 100% duplication but also not 0% abstractions.
- piva00 3y ago
  > I guess the "truth" or "optimal state" is somewhere in-between. Not 100% duplication but also not 0% abstractions. Exactly, I've seen way too much test code that abstracted away a fundamental piece needed for understanding the test setup, which made it much harder to reason about when tests failed. Jumping through the test to the utility used to avoid duplication is another layer of cognitive load, I'm already trying to troubleshoot something, I don't need more things to keep track of while I'm in that process. Rewriting test code is exactly the moment you take to abstract some of it away, when it's been proven that the pattern repeats and is annoying to rewrite. The usual rule of 3 of software engineering. > it's also important that you're able to change the test code confidently, and duplication actively works against that. I disagree with this point a little, if the tests are well written (even if verbose) it should be easy to reconfigure/rewrite them, even with dozens of test cases, it might take a bit more time to rewrite those than just changing a supplier/utility method but it has helped me a lot to actually read the tests. I've been burnt too many times by someone abstracting away the actual call (or a part of it) to the logic under test into an utility to "avoid duplication", that call is the whole point of the test case. One should be able to understand every test case easily so rewriting to accommodate the change made should be straightforward, if it's not then the test case itself is telling there's a code smell, either in the test or in what is under test. The more egregious to me is when tests are a mess of setups not following a simple, straightforward structure. It becomes hard to reason and change over time when the inevitability of growing code happens. Duplication or not won't be the main issue, the structure itself will. I will abstract away some complex setup/adjacent code that isn't under test but anything that the test cases might interact with, and fail because of it I do prefer to be extremely explicit and duplicate code instead of trying to make my tests look tidy and neat.