3 ms·

Your issue is that you're not mocking correctly. You need to write mocks for services you depend on, ideally by recording and replaying real api interactions, s

by TheDong 12y ago

Your issue is that you're not mocking correctly. You need to write mocks for services you depend on, ideally by recording and replaying real api interactions, such that you can verify that given the libraries you depend on your tests will be correct. Then write TDD tests, then write code, then write integ tests which will verify your original mocks work as assumed.

millstone 12y ago
When your software is tightly coupled to some complicated external component, the most likely cause of bugs is bad assumptions about how that component behaves. The integration tests are therefore the most important, since those are what will expose the bad assumptions. I wrote a program that used the computer's MAC address. The API that reports the MAC strips leading 0s, but my MAC had no leading 0s and so I I did not notice this behavior during development. If I had just captured and replayed this result, my tests would have passed on all machines, even affected ones. But because my tests exercised the real API, I encountered the bug and was able to work around it.
mikekchar 12y ago
Another approach (which is not really so different) is to build adapters around external services and then write your code to use the adapters. The upside is that you can avoid writing integration tests which might be very difficult to set up (depending on the situation). The downside is that you might implement more in the adapter than you need. In practice, I tend to do a mixture. I will mock/fake the external services and then build adapters and then finally remove the mocks/fakes. In the final part you may have to stub the adapters. The unit tests in the adapters will alert me to changes in the external services, so this is relatively safe to do. Your main sources of error will be subtle bugs where you have insufficient coverage in the adapters. You must also use restraint in working on the adapters independent of the code that uses it, because you have no end to end tests. Building fakes instead of mocks in this situation is also a good way to really test your understanding of the external services. It takes more time, but it can pay dividends. Many times I will write a fake and then think, "Does it really work that way?" only to realize that I can't actually use the external service in the way I envisioned. With a mock, you completely divorce yourself from implementation so it is relatively easy to assume that the external service can do impossible things. You can go a long way before you realize that your code is unusable. One thing I will caution -- mocks, fakes, stubs and integration tests tend to attract people with almost religious views in how they should be used. I tend to believe that this is stuff is hard and that current industry practice is very naive. I think there is a lot of room for experimentation with a very big upside. But you are likely to attract a lot of criticism if you wander outside someone's boundaries, so it is best to spend significant amounts of time pairing to make sure that everybody is comfortable with the approaches you use. Nothing is worse that "test wars" where people are going in 100 different directions in the tests.
- rjbwork 12y ago
  This is the approach we take: Adapters+mocks for external services, but we're okay with hard dependencies (via injection if possible) on non-network libraries. This includes things like json serializers, other serializers, automappers, math libraries, etc.
cowardlydragon 12y ago
If you're doing extensive mocking, then you should be doing integration tests instead.

by TheDong 12y ago

millstone 12y ago

When your software is tightly coupled to some complicated external component, the most likely cause of bugs is bad assumptions about how that component behaves. The integration tests are therefore the most important, since those are what will expose the bad assumptions. I wrote a program that used the computer's MAC address. The API that reports the MAC strips leading 0s, but my MAC had no leading 0s and so I I did not notice this behavior during development. If I had just captured and replayed this result, my tests would have passed on all machines, even affected ones. But because my tests exercised the real API, I encountered the bug and was able to work around it.

mikekchar 12y ago

Another approach (which is not really so different) is to build adapters around external services and then write your code to use the adapters. The upside is that you can avoid writing integration tests which might be very difficult to set up (depending on the situation). The downside is that you might implement more in the adapter than you need. In practice, I tend to do a mixture. I will mock/fake the external services and then build adapters and then finally remove the mocks/fakes. In the final part you may have to stub the adapters. The unit tests in the adapters will alert me to changes in the external services, so this is relatively safe to do. Your main sources of error will be subtle bugs where you have insufficient coverage in the adapters. You must also use restraint in working on the adapters independent of the code that uses it, because you have no end to end tests. Building fakes instead of mocks in this situation is also a good way to really test your understanding of the external services. It takes more time, but it can pay dividends. Many times I will write a fake and then think, "Does it really work that way?" only to realize that I can't actually use the external service in the way I envisioned. With a mock, you completely divorce yourself from implementation so it is relatively easy to assume that the external service can do impossible things. You can go a long way before you realize that your code is unusable. One thing I will caution -- mocks, fakes, stubs and integration tests tend to attract people with almost religious views in how they should be used. I tend to believe that this is stuff is hard and that current industry practice is very naive. I think there is a lot of room for experimentation with a very big upside. But you are likely to attract a lot of criticism if you wander outside someone's boundaries, so it is best to spend significant amounts of time pairing to make sure that everybody is comfortable with the approaches you use. Nothing is worse that "test wars" where people are going in 100 different directions in the tests.

rjbwork 12y ago
This is the approach we take: Adapters+mocks for external services, but we're okay with hard dependencies (via injection if possible) on non-network libraries. This includes things like json serializers, other serializers, automappers, math libraries, etc.

cowardlydragon 12y ago

If you're doing extensive mocking, then you should be doing integration tests instead.