3 ms·

[disclaimer: I work at https://circleci.com https://circleci.com, so I do have a vested interest in disagreeing with you ;) ] I think is a strawman. If running

• puredanger 13y ago
  Hey Paul, I totally agree that the cost of running big test suites can be mitigated and CircleCI is doing awesome stuff there. That does nothing to address the problem of maintaining a large suite of tests tightly coupled to your code. That is, the cost of updating many example-based tests when you change the code.
  • pbiggar 13y ago
    I guess that depends on how much of your suite is in flux. We have largely example-driven tests, and our suite is pretty big: about 16kLOC. I can't think of a case where a significant amount of time was spent fixing the test suite (and in those cases, it was generally useful to think about why). Also, I haven't seen anything that works better. Generative tests are nice in theory, but I haven't seem the be that useful for anything other than short, pure, functions.
    • jamii 13y ago
      I've built generative tests for an entire betting exchange. We run the database in-memory, throw generated events in one end and make assertions about the state when processing is finished. The tests range from very specific, narrow pseudo-unit-tests to coarse properties like 'if we process 1000 random orders from 80 different customers in 3 different currencies, the total amount of money in the system remains constant'. I can't imagine reaching the same level of stability in the core exchange if we had to write all the test cases by hand.