4 ms·
When you build and sell a network security appliance, the appliance has to work properly all the time. You have to give guarantees of how it will perform, guara
by throwaway984393 5y ago
When you build and sell a network security appliance, the appliance has to work properly all the time. You have to give guarantees of how it will perform, guarantee that its functions do what you claim they do, and eliminate all possible bugs. So you have to test it in every configuration possible.
So you make models and build automation test frameworks and labs full of prototype equipment so you can run 10,000 tests an hour, 24/7. You automate setting up networks and nodes and passing live traffic to test detectors and rule sets. You use custom hardware to emulate ISP-levels of real traffic.
You can't really mock anything. You have to be sure each function will perform as you describe. So most of this testing is end-to-end testing. Using a mock wouldn't tell you if the millions of code paths are working correctly; they'd mostly just tell you if the syntax/arguments/etc of a call were correct. Unit tests are basically one step above evaluating that your code compiles correctly, but it's not testing the code works as expected. End-to-end tests are what you can rely on, because it's real traffic to real devices.
That's gonna look different for a lot of your apps, but for web apps that means getting real familiar with synthetic tests and how to test with different browsers. For SDKs it means end-to-end tests for the whole stack of functions, which is a lot more testing than you may have expected. For APIs it means getting the consumers of your APIs involved in end-to-end testing. It also means "testing in production", in the sense of spinning up new production deployments and using those as your end-to-end testing targets (rather than having a dedicated "dev/test/cert" environment which is always deviating from production). This can take a significant effort to adapt in legacy systems, but for Greenfield IaC projects is not very hard to set up.