3 ms·

Everyone talking about unit tests, and testing in general as a thing to achieve are missing it entirely. Tests aren't a thing to achieve, and talking about whet

Everyone talking about unit tests, and testing in general as a thing to achieve are missing it entirely. Tests aren't a thing to achieve, and talking about whether unit testing, mocks etc. are better or worse without context is pointless. The thing to achieve is having a code base that 1) works 2) can be changed easily and proved to be still working. Now that we know that we can start deciding "how" we can achieve that and it will be different for every code base (and often different strategies for different parts of the code base). The key thing to realize in a test strategy is that your code has two interfaces, not one. The first interface is the user, the second is some form of automated test harness. Hence, testing becomes a function of two points above and a design required to achieve that. In summary, don't do this: - I need 100% code coverage - Everything needs a mock - Ensuring unit tests for everything Do this: - Ask how can I ensure my code works? - How can I ensure my code keeps working? - What sort of testing strategy can achieve the above? - What sort of testing strategy is easily maintained without causing tons of excess burden? - What interfaces do I need on each module of my software to achieve this?

zbentley 2y ago
Very good advice. Additionally, I'd add: - What sort of testing can be achieved given limited time and resources? ...because nearly all organizations, no matter how quality-oriented they claim to be, will prioritize non-test code delivery over test code, leaving scarce time for test creation before shipment deadlines.
- farley13 2y ago
  Not to pick on you specifically, but I tend to agree with other posters that testing (automated or otherwise) is just an element of programming. Like all elements it needs to be done to taste, but it's pretty essential. A line of questioning: Do you have time to write clear code? Time for comments? Time to manually test your changes hundreds of times? Time to refactor existing code when adding new code? Time to remove dead code? Time to automate the testing while you still have the little state machine you are working on in your head? Time to add observability to spot performance issues? Time to consider your rollout plan? Time to keep on top of changes which are in use? etc... Automated tests (including unit tests) are just one part of writing correct code. If you are asked, "How long is it going to take?" that implies finishing all aspects of coding required to get something correct into use. You prioritize that, not anyone else.
  - zbentley 2y ago
    I fully agree with all your points. My comment was based in my observations that, even given the truth of everything you said, "time to prepare automated tests" is usually one of the first things to shrink as unexpected events cause projects to slip on their timelines. That's not a good thing, but it is a real thing: healthy organizations should respond by protecting that time, extending deadlines, and assessing what about their process and environment is causing planned timelines to not match reality ... but that doesn't always happen. When it doesn't happen, testing discipline can slip from (for example) "we need unit tests for complex logic modules, and integration tests for real networked service interactions, and at least 70% coverage" to "just get the happy path tested however you can so this feature makes it into the next release". Given the reality that this will sometimes occur, engineers should remember to always prioritize the highest-value (that is, the maxima between time spent creating tests and defects that those tests can catch) testing work and methodologies. In good conditions, that will result in them writing the most important tests first and then writing any other tests they feel they need. But in bad conditions, this approach will still ensure that the most important automated verification is present.
luisgvv 2y ago
I wish management knew this... It's a nightmare when some manager that hasn't coded in years thinks it's a good idea to push unit testing into the CI builds so that it fails if it doesn't meet code coverage criteria etc.
kikimora 2y ago
I afraid in practice answering latter questions brings you to the former statements.

zbentley 2y ago

Very good advice. Additionally, I'd add: - What sort of testing can be achieved given limited time and resources? ...because nearly all organizations, no matter how quality-oriented they claim to be, will prioritize non-test code delivery over test code, leaving scarce time for test creation before shipment deadlines.

farley13 2y ago
Not to pick on you specifically, but I tend to agree with other posters that testing (automated or otherwise) is just an element of programming. Like all elements it needs to be done to taste, but it's pretty essential. A line of questioning: Do you have time to write clear code? Time for comments? Time to manually test your changes hundreds of times? Time to refactor existing code when adding new code? Time to remove dead code? Time to automate the testing while you still have the little state machine you are working on in your head? Time to add observability to spot performance issues? Time to consider your rollout plan? Time to keep on top of changes which are in use? etc... Automated tests (including unit tests) are just one part of writing correct code. If you are asked, "How long is it going to take?" that implies finishing all aspects of coding required to get something correct into use. You prioritize that, not anyone else.
- zbentley 2y ago
  I fully agree with all your points. My comment was based in my observations that, even given the truth of everything you said, "time to prepare automated tests" is usually one of the first things to shrink as unexpected events cause projects to slip on their timelines. That's not a good thing, but it is a real thing: healthy organizations should respond by protecting that time, extending deadlines, and assessing what about their process and environment is causing planned timelines to not match reality ... but that doesn't always happen. When it doesn't happen, testing discipline can slip from (for example) "we need unit tests for complex logic modules, and integration tests for real networked service interactions, and at least 70% coverage" to "just get the happy path tested however you can so this feature makes it into the next release". Given the reality that this will sometimes occur, engineers should remember to always prioritize the highest-value (that is, the maxima between time spent creating tests and defects that those tests can catch) testing work and methodologies. In good conditions, that will result in them writing the most important tests first and then writing any other tests they feel they need. But in bad conditions, this approach will still ensure that the most important automated verification is present.

luisgvv 2y ago

I wish management knew this... It's a nightmare when some manager that hasn't coded in years thinks it's a good idea to push unit testing into the CI builds so that it fails if it doesn't meet code coverage criteria etc.

kikimora 2y ago

I afraid in practice answering latter questions brings you to the former statements.