6 ms·
Unit testing, mocking, and various other testing techniques. Why? Any project of sufficient complexity is very hard to test. If all you're doing is code -> bui
by gwbas1c 8y ago
Unit testing, mocking, and various other testing techniques.
Why? Any project of sufficient complexity is very hard to test. If all you're doing is code -> build -> run to debug your code, you can very easily break something that's not in your immediate attention.
The problem is that good unit testing is hard, and time consuming. It can be so time consuming, that unless you can really plan in advance how you test, you could spend more time writing test code than real application code. (This is what happens when writing professional, industrial-strength code.)
So, when a hobby project becomes sufficiently interesting enough; such that the code will be so complicated that your code -> build -> run loop won't hit most of your code, you should think about how to have automated tests. They don't have to be "pure, by the book" unit tests, but they should be an approach that can hit most of your program in an automated manner.
You don't need to do "pure" mocking either. If you're writing something that calls a webservice, you could write a mock webserver and redirect your program to it. If you're writing something that works with pipes, you could have a set of known files with known results, and always compare them.
The goal is that you should cover most of your program with code -> build -> tests; and only do code -> build -> run for experimentation.
- wpietri 8y agoLet me second this. And in particular, I strongly encourage every developer to try starting a new project in a test-driven fashion (by which I mean that you advance the code by writing a bit of test and making it pass, and then doing that over and over.) There's a qualitative difference between working in a well-tested code base that's very hard to describe convincingly. A lot of my early development experience was in code bases that had little or no testing. Experiencing a well-tested code base totally changed things for me. Instead of work being a death-of-a-thousand-cuts experience, it became pleasant, steady progress.
- mooreds 8y ago> Experiencing a well-tested code base totally changed things for me. Instead of work being a death-of-a-thousand-cuts experience, it became pleasant, steady progress. I had the luxury of taking a well known data process and rewriting it with integration tests (input in, matching output with a golden file). It changed my professional life. Whereas before our deployment process included a 3 day wait and manual data checking on stage, after I was able to do deploys multiple times a day with confidence. Made a believer out of me.
- JamesBarney 8y agoUnit tests can give you false positives (test failed but code is correct) and false negatives (test passed but code failed). And TDD seems to create so many tests that you get huge false positive rates. I recently jumped on a project and I made a couple of fairly small code changes (a couple of hours) which caused 100 tests to fail. I then spent the next two days going through and correcting all 100 tests none of which found an issue in my code.
- ryanmclovin 8y agoDoes that indicate that the tests were not written correctly in the first place?
- JamesBarney 8y agoI don't think so. They probably could have been written better, they weren't written poorly, but it's really hard to write 200 unit tests for a feature that don't break when the feature is updated.
- nostrebored 8y agoThe type of engineers who would screw up 100 unit tests independently are exactly the kind of engineers who should be forced to write tests for their code. Can you imagine the integration tests had they not been doing any testing at all?
- wpietri 8y agoIf you're saying that it's possible to do testing badly, I agree, just like it's possible to write production code badly. Sometimes teams new to unit testing do it ritualistically, without really understanding the purpose. That can lead to all sorts of bad outcomes. E.g., lots of tests that look impressive and even generate good coverage numbers, but don't really test what matters. Or tests that are highly duplicative, such that changing one thing in the code requires changing a lot of things in tests. I have definitely dealt with code bases like that, and that sucks. But I have also dealt with code bases where the tests were great, and that's an amazing experience. To do TDD well, I think it's important to release early and often and to reflect on one's experience (e.g., with weekly team retrospectives). That way if people are doing something unhelpful, like writing very duplicative tests, pretty soon they'll become an impediment to progress. The team will learn to write the useful tests, while skipping the ones that might fit some hypothetical pattern. It also helps people learn to design for testability; often, painful tests are a sign of bad design of the production code.
- epiphanitus 8y agoAre there any resources out there you would recommend for learning testing techniques in a Python context?
- sn9 8y agoTDD with Python: https://www.obeythetestinggoat.com/ https://www.obeythetestinggoat.com/
- heyoni 8y agoIs that useful if you’re never writing django apps?
- a_c 8y agoIt depends on your background. Having written web app before let's you quick grasp the ideas laid in the book. To me, the most important chapters are - https://www.obeythetestinggoat.com/book/chapter_mocking.html https://www.obeythetestinggoat.com/book/chapter_mocking.html - https://www.obeythetestinggoat.com/book/chapter_purist_unit_tests.html https://www.obeythetestinggoat.com/book/chapter_purist_unit_... Having said that, the concepts are universal.
- bklaasen 8y agoBrian Okken's "Python Testing with pytest"[1]. More recent than Harry Percival's book. [1] https://pragprog.com/book/bopytest/python-testing-with-pytest https://pragprog.com/book/bopytest/python-testing-with-pytes...
- lenticular 8y agoThis is the gospel truth. It does take discipline though, because writing tests sucks. I like to have a policy of never committing a non-trivial function without a test. That way, I can never put it off and wind up with a huge chunk of untested code.
- crimsonalucard 8y agoYou are completely wrong. Mocking is a huge design smell. The more mocks or integration tests your projects requires to get full coverage the less modular your program is. A program that uses many mocks is a sign of very very poor design. You will find the code more complex to reason about and much harder to reuse code without necessitating a lot of glue code to make things work together. Without proper knowledge you won't even know the program is poorly designed. I will grant you that 90% of programmers out there don't know how to design programs in a truly modular way, so most engineering projects will require extensive mocking. In fact most engineers can go through their entire career without knowing that they are making their programs more complex and less modular then it needs to be. Following certain design principles I have seen incredibly complex projects require nearly zero mocking (very very rare though). Mocking indicates a module is dependent on something. Dependency is different from composition. Dependencies Composition C C +---------------------+ | | +----------------+ +-----------------+ | A | | | | | | | | | | | | +----------+ | | | | | | | | | in | | | | out | | | | -->+ A +------>+ B +--> | | B | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +----------+ | | | | | +---------------------+ +----------------+ +-----------------+ What's going on here? Both examples involve the creation of module C from A and B. left: 'A' exists as wrapper code around B and is useless on its own. To unit test A you must mock B. right: every module is reuseable on its own. Nothing needs to be mocked during unit testing. No dependencies. The only exception to the right example where you MUST mock is a function that does IO. IO functions cannot be unit tested period, they can only be tested with integration tests. There's a name for the left approach. It's called Object oriented programming using inheritance or composition(the oop version of composition; not functional composition) as a design pattern. (both are bad) There's also a name for the right approach. It's called functional programming using function composition. I don't advocate that you strictly follow either style. Just know that when you go left you lose modularity and when you go right you gain it. All functional programming does is force your entire program to be modular down to the smallest primitive unit. Extensive mocking in your program means you went too far to the left. tangent: Another irony around this world is that a lot of functional programmers (javascript and react developers especially) don't even know about the primary benefit of functional programming. They harp about things like "immutability" or how its more convenient to write a map reduce rather than a for loop without truly ever knowing the real benefits of the style. They're just following the latest buzzword.