3 ms·

3. Your code base may start to become contorted. I've seen good programmers create bogus classes to allow test-time mocking, or add oddball env vars and configu

3. Your code base may start to become contorted. I've seen good programmers create bogus classes to allow test-time mocking, or add oddball env vars and configurations to let the test harness manually reach every last line. Even if that line is not worth testing: if(!(x=malloc(BUF_SIZ)) || ENV[TEST_MEM_FAIL_12]) { exit(1); } Tying code and tests this tightly discourages refactoring. Another example: a different code base that I helped on had some insanely long one-line conditionals that anyone would break into multiple lines ... but doing that would have blown the "100%" coverage.

hsn915 5y ago
"Mocking" is such a weird thing to me and I don't think it serves a good purpose. It's the kind of thing that would only arise if you assume a-priori that 100% test coverage is a non-negotiable must. If you have a function A that calls B to get some data (by doing I/O) then process it using C, then the 100% cov rule would force you to mock B when you test A. But then what is the value of this test? What guarantees is it giving you? You are basically testing the internals of A, you are testing the implementation details. You are testing that calling A is the same as calling B and passing the result to C.
- berkes 5y ago
  Mocking helps with architecture, though. When A calls C through B, you have a tight coupling between A and C, but mightn't be aware of that, because you only see A calling B. Lets say C reads a CSV file. By testing implementation only, you might 'create a csv file' then call A.import() and assert some records in, say D are created. By testing A in isolation, you make apparent that it couples to C and D because you have to mock them out. B is a direct dependency of A, so you don't mock that. At the least, you have now documented the accidental coupling. But at the best, your tests showed you a design issue.
- chriswarbo 5y ago
  Mocking and coverage aren't that closely related, but I agree that mocking makes tests less realistic, and therefore less useful. Mocking can be helpful when that downside is still better than the alternative, e.g. if 'real' calls have significant latency, or a significant monetary cost, etc. In those cases mocking lets us do a whole lot more testing for the same amount of time/money/etc., which can make up for the loss of realism. Mocking can be especially useful when adding tests to a legacy system. Greenfield work should aim to minimise the amount of mocking required, ideally to zero.

hsn915 5y ago

"Mocking" is such a weird thing to me and I don't think it serves a good purpose. It's the kind of thing that would only arise if you assume a-priori that 100% test coverage is a non-negotiable must. If you have a function A that calls B to get some data (by doing I/O) then process it using C, then the 100% cov rule would force you to mock B when you test A. But then what is the value of this test? What guarantees is it giving you? You are basically testing the internals of A, you are testing the implementation details. You are testing that calling A is the same as calling B and passing the result to C.

berkes 5y ago
Mocking helps with architecture, though. When A calls C through B, you have a tight coupling between A and C, but mightn't be aware of that, because you only see A calling B. Lets say C reads a CSV file. By testing implementation only, you might 'create a csv file' then call A.import() and assert some records in, say D are created. By testing A in isolation, you make apparent that it couples to C and D because you have to mock them out. B is a direct dependency of A, so you don't mock that. At the least, you have now documented the accidental coupling. But at the best, your tests showed you a design issue.
chriswarbo 5y ago
Mocking and coverage aren't that closely related, but I agree that mocking makes tests less realistic, and therefore less useful. Mocking can be helpful when that downside is still better than the alternative, e.g. if 'real' calls have significant latency, or a significant monetary cost, etc. In those cases mocking lets us do a whole lot more testing for the same amount of time/money/etc., which can make up for the loss of realism. Mocking can be especially useful when adding tests to a legacy system. Greenfield work should aim to minimise the amount of mocking required, ideally to zero.