4 ms·
Hey Paul, I totally agree that the cost of running big test suites can be mitigated and CircleCI is doing awesome stuff there. That does nothing to address the
by puredanger 13y ago
Hey Paul, I totally agree that the cost of running big test suites can be mitigated and CircleCI is doing awesome stuff there.
That does nothing to address the problem of maintaining a large suite of tests tightly coupled to your code. That is, the cost of updating many example-based tests when you change the code.
- pbiggar 13y agoI guess that depends on how much of your suite is in flux. We have largely example-driven tests, and our suite is pretty big: about 16kLOC. I can't think of a case where a significant amount of time was spent fixing the test suite (and in those cases, it was generally useful to think about why). Also, I haven't seen anything that works better. Generative tests are nice in theory, but I haven't seem the be that useful for anything other than short, pure, functions.
- jamii 13y agoI've built generative tests for an entire betting exchange. We run the database in-memory, throw generated events in one end and make assertions about the state when processing is finished. The tests range from very specific, narrow pseudo-unit-tests to coarse properties like 'if we process 1000 random orders from 80 different customers in 3 different currencies, the total amount of money in the system remains constant'. I can't imagine reaching the same level of stability in the core exchange if we had to write all the test cases by hand.