4 ms·
I've had problems relying heavily on functional/integration tests. The first problem is data consistency. Both require consistent data in whichever environment
by nahname 15y ago
I've had problems relying heavily on functional/integration tests.
The first problem is data consistency. Both require consistent data in whichever environment they are running. Not an insurmountable goal, but one that often leads to a lot of false failures. These lead to a lack of trust in the test feedback.
The second problem, as others have already pointed out, is that integration tests and functional tests don't tell you what is failing or why. Can I create a user? Nope. Why? 500 error. Time to fire up the debugger...
The third problem is that functional tests do not scale. Each functional test will take orders of magnitude longer to run than integration tests, which take orders of magnitude longer than unit tests. You very quickly end up in the situation where you have more hardware than people or you no longer get fast feedback.
Personally, feedback is the primary driver behind test automation. It is nice to know what I wrote will probably work and it certainly helps simplify the code. However, the biggest pay off comes from the fast and repeatable verification that what I changed has not impacted the code base negatively.