3 ms·

"An analytics system without tests that generated incorrect results silently." I've noticed that test coverage and quality is largely a people (budget) problem

"An analytics system without tests that generated incorrect results silently." I've noticed that test coverage and quality is largely a people (budget) problem. They want the system now. They want that value delivered number going up and the cycle time going down. Tests are some of the first cuts.

convolvatron 2y ago
i think thats too simple. i'm older, and when i go into shops its not that everyone realizes they need testing and it got cut, its that no one even has any notion that testing is useful or interesting. an entire generation of developers propping up systems without any real systemic validation that they do what they are supposed to. organizations spend a huge amount of resource keeping 'cicd' propped up, but when you look at whats inside - its a build, and some random unit tests and mocks. to be fair, testing in an environment where everything is internal and external services laced together with json isn't easy. what i find really odd is just that no one is willing to acknowledge it as a real problem.
- giantg2 2y ago
  That might be true in some places. In my company, they've systemically cut QA positions and testing gets pushed off. We spend about 25% of our time fixing bugs and defects every sprint. They are still pushing us for more speed.
- rectang 2y ago
  Even so, the difference between some tests and no tests is vast. It's worth keeping CI/CD "propped up" just to detect syntax/compilation errors if nothing else. (And then those of us who care at least have the option of writing some tests. I'll take the heat for being "slower" if I have to, because a bare minimum of tests provides extremely high value and advocating for baseline CI/CD is a great way to contribute to any organization.)
bunderbunder 2y ago
I've seen tests cut for that reason. But I've more often seen tests get cut or skimped on or slip through the cracks due to internal team dynamics problems. If QA and engineering staff are organized into separate verticals, rather than being direct teammates that share a manager, then time that could have been spent on testing instead gets spent on bickering over who tests what and how. If different team members have different opinions on testing strategy, and dig in on their respective positions rather than coming to an understanding, then tests will slip through the gap between them. If senior team members don't think it's their responsibility to mentor (not browbeat!) less-experienced team members on good testing practice, then the correct tests won't get written. If the people in charge of architecture don't recognize the outsize influence they have on the cost of designing and implementing tests, then testing can indeed become too expensive. If people don't recognize that test cases are, first and foremost, a way of communicating and memorializing design specifications, and not just a clever way to run code in isolation, then the cost of maintaining them will go up and the value of maintaining them will go down, until people start to forget that there was ever a baby in all that bathwater in the first place. etc.

convolvatron 2y ago

i think thats too simple. i'm older, and when i go into shops its not that everyone realizes they need testing and it got cut, its that no one even has any notion that testing is useful or interesting. an entire generation of developers propping up systems without any real systemic validation that they do what they are supposed to. organizations spend a huge amount of resource keeping 'cicd' propped up, but when you look at whats inside - its a build, and some random unit tests and mocks. to be fair, testing in an environment where everything is internal and external services laced together with json isn't easy. what i find really odd is just that no one is willing to acknowledge it as a real problem.

giantg2 2y ago
That might be true in some places. In my company, they've systemically cut QA positions and testing gets pushed off. We spend about 25% of our time fixing bugs and defects every sprint. They are still pushing us for more speed.
rectang 2y ago
Even so, the difference between some tests and no tests is vast. It's worth keeping CI/CD "propped up" just to detect syntax/compilation errors if nothing else. (And then those of us who care at least have the option of writing some tests. I'll take the heat for being "slower" if I have to, because a bare minimum of tests provides extremely high value and advocating for baseline CI/CD is a great way to contribute to any organization.)

bunderbunder 2y ago

I've seen tests cut for that reason. But I've more often seen tests get cut or skimped on or slip through the cracks due to internal team dynamics problems. If QA and engineering staff are organized into separate verticals, rather than being direct teammates that share a manager, then time that could have been spent on testing instead gets spent on bickering over who tests what and how. If different team members have different opinions on testing strategy, and dig in on their respective positions rather than coming to an understanding, then tests will slip through the gap between them. If senior team members don't think it's their responsibility to mentor (not browbeat!) less-experienced team members on good testing practice, then the correct tests won't get written. If the people in charge of architecture don't recognize the outsize influence they have on the cost of designing and implementing tests, then testing can indeed become too expensive. If people don't recognize that test cases are, first and foremost, a way of communicating and memorializing design specifications, and not just a clever way to run code in isolation, then the cost of maintaining them will go up and the value of maintaining them will go down, until people start to forget that there was ever a baby in all that bathwater in the first place. etc.