3 ms·
Disclaimer: I only took a quick glance at the material, not a deep dive. So feel free to discount my criticism as severely as you like. Or whack me with a clu
by dbcurtis 8y ago
Disclaimer: I only took a quick glance at the material, not a deep dive. So feel free to discount my criticism as severely as you like. Or whack me with a clue-bat, if needed.
1. In the fuzzing chapter author seems to be showing how to create a fuzzer in Python. OK, so that is great if you want to learn the basics of creating a fuzzer. HOWEVER... there is a great Python module called "hypothesis" that is well developed, well maintained, and 100% production ready. If your goal is to do fuzzed testing of Python, I urge you to check out hypothesis. It integrates nicely with drivers like unittest, and does automatic test case reduction. In other words, if it discovers a failing case, it does a pretty darn decent job of trimming the failing case down to the minimal inputs necessary to provoke the failure. In fact, I urge you to check out hypothesis, period. You might be surprised at how much your current testing leaves out.
2. The first chapter on testing talks about the basics of doing a unit test and writing a test driver. Again, test drivers that are well maintained and have a lot of mileage on them already exist for Python. But more over, the author seems to focus on tactical testing. Unit testing is good, yes, but there is a difference between "testing" and "validation". Testing is tactical. Validation is strategic. I didn't see anything there about analyzing the product test space, and how to make judgment calls about how to decide what to test and how to measure coverage in a way that ensures that your customers will be well-served. For most real products, the entirety of the test space is intractably large, so it is very important to know how to target tests that tickle edge and corner cases, because just throwing random crap at the SUT, while necessary, is unlikely to sensitize the interesting test points. (Imagine each test case as throwing darts at a dart board: You want as many darts to land on the line between the rings as land cleanly within the rings.)