The algorithms we were implementing were very hard to test using conventional unit test techniques. The only way to properly test this code was to run the full

The algorithms we were implementing were very hard to test using conventional unit test techniques. The only way to properly test this code was to run the full system with recorded input data. We used data extractions at key points in the pipeline to verify that the code was working correctly. With a series of extractions it was possible to figure out where the problem was, and with an understanding of the algorithms it was possible to figure out what it was. The extractions themselves could be placed at arbitrary locations.