4 ms·
One option is to simply write end to end tests, and not bother with unit tests at all. This does mean that you don't know if tests are failing because of a new
by archgoon 6y ago
One option is to simply write end to end tests, and not bother with unit tests at all. This does mean that you don't know if tests are failing because of a new code change or if the world has changed from underneath you.
If you want to write a unit test, then the test would mock the response from the server. You would have an expected sequence of expected requests and their responses. You would only be testing that you made the correct requests and, given the same response from the server, your code parsed it correctly and gave the expected outputs.
If the server bans your IP, or changes their response codes, your unit tests will still pass, as they should. Notification that the world has changed around you is not the purpose of a unit test. It is to allow you to feel confident of claims "I refactored the code and behavior is unchanged." or "I changed the behavior and this is how it has changed" or "I know that my code handles all known error codes from the server."
This is important when the world does change around you; as it lets you identify when that happens.
I recognize that test terminology is vague, and people will use competing definitions (I have seen 'functional', 'integration', and 'end to end' all used to describe non-unit tests, and I've seen each used to mean the others). I recognize this may seem as an exercise in pedantry. However, there is a valuable distinction to be made between tests that can be run deterministically, and tests that rely on the behavior of third parties outside of your control. These tests types will differ in terms of speed of execution, infrastructure setup (many test environments may not have network access, or may need special credential access)[1], reliability, and also, importantly, the proper response to a failure.
[1] Also, if you're doing mobile app development, you very much want to distinguish between tests that require a device and ones that don't. You will likely have multiple test classifications if you're doing mobile development.
- djeiasbsbo 6y agoI guess the difference with youtube-dl is that there can be a change in the code base aswell as a server side change. The unit tests should only cover changes in the codebase... probing whether something serverside changed and whether the source material can still be downloaded can also be automated, but preferably not in a pipeline triggered by commits.