3 ms·

Integration tests can (and often do) suffer from the same issues since being composed of poorly-testable pieces doesn’t make things any more testable. I’ve com

Integration tests can (and often do) suffer from the same issues since being composed of poorly-testable pieces doesn’t make things any more testable. I’ve come to the conclusion that mocks at all are evil when done for anything except external services not under your control. And then, what you should be making is a trivial fake implementation of the other end, not just checking that specific methods were called.

taeric 2y ago
I mean, anything can be done poorly. But consider that the team that builds the vehicles you drive is not the same as the team that tested said vehicle for safety concerns. That is, you can have a solid testing team for pretty much anything. You have to empower them, though. Largely, a lot of what you get to empower them with is ability to put constraints onto the engineering team. This could be crappy constraints that are far sweeping on the product. There is no need that it has to be that, though. The only constraint you have to have is stability of product interface. Which.... yeah, our industry doesn't do that so well. (Note, not stability as in "doesn't crash." Stability as in, same inputs that worked last year work today.)
RandomThoughts3 2y ago
When I was doing safety critical software at the beginning of my career, we had a whole team in charge of building simulators replicating the API provided by the pieces of equipment we were integrating and their full behaviour. These simulators were themselves fully tested using both mock network traffic and recording of the real piece of equipment. This was used for integration testing and then repurposed for training. Every testing environment I have seen since have looked like a joke but to be fair the testing budget of this place was bigger than what some companies spend on a whole product.

3 ms·

Integration tests can (and often do) suffer from the same issues since being composed of poorly-testable pieces doesn’t make things any more testable. I’ve com

by stouset 2y ago

taeric 2y ago
I mean, anything can be done poorly. But consider that the team that builds the vehicles you drive is not the same as the team that tested said vehicle for safety concerns. That is, you can have a solid testing team for pretty much anything. You have to empower them, though. Largely, a lot of what you get to empower them with is ability to put constraints onto the engineering team. This could be crappy constraints that are far sweeping on the product. There is no need that it has to be that, though. The only constraint you have to have is stability of product interface. Which.... yeah, our industry doesn't do that so well. (Note, not stability as in "doesn't crash." Stability as in, same inputs that worked last year work today.)
RandomThoughts3 2y ago
When I was doing safety critical software at the beginning of my career, we had a whole team in charge of building simulators replicating the API provided by the pieces of equipment we were integrating and their full behaviour. These simulators were themselves fully tested using both mock network traffic and recording of the real piece of equipment. This was used for integration testing and then repurposed for training. Every testing environment I have seen since have looked like a joke but to be fair the testing budget of this place was bigger than what some companies spend on a whole product.