4 ms·
Yeah, we've made the mistake of testing too much in our last configuration management. A test suite can be strangling if it starts to restrict every single chan
by tetha 4y ago
Yeah, we've made the mistake of testing too much in our last configuration management. A test suite can be strangling if it starts to restrict every single change to the code base, unless you touch like 3-4 tests per configuration file deployed to a server.
At this point, most of our tests are much higher level by default - setup 3 nodes, run the config management against it, check if the cluster formed. If the cluster formed, the setup from the config management can't be that wrong.
From there, we're adding tests in an outage/bug-driven way. If it doesn't deploy and load a config correctly into the application, that's a test to write. Or, we recently almost ran into an outage because some failure earlier on caused the config management to start tearing the cluster apart because of missing metadata. So we added a test that it cancels the rollout in such a case.
Those tests are valuable, because we can rely on these issues not cropping up again. Some test breaking on yaml formatting isn't as much.
- Veuxdo 4y agoAre your tests for "the configuration is correct" or "the configuration file was found and read"...?
- tetha 4y agoIn our old config management, we would indeed go ahead and parse configuration files and check values within the configuration files - even values entirely unrelated to the test at hand. Then you'd go ahead and change some default value in the config management because now you understand the system more and a whole bunch of unrelated tests would start failing. By now, we mostly do a quick syntax check of the config and the rest is usually handled by smoke-testing the application. The config files are usually only parsed if they contain some non-trivial degree of dynamic computation.