> why not check for full equality with a pre-existing pdf file, byte for byte? This was what I built once to do unit tests on a pdf generator. The use case wa

> why not check for full equality with a pre-existing pdf file, byte for byte? This was what I built once to do unit tests on a pdf generator. The use case was I was working for a software vendor at a very important financial client. Our software was used in their research group to produce output (massive reports) which fed into their trading decisions and had been heavily customized by the client in ways that our dev team couldn’t test (the client had a very exacting internal confidentiality regime and these reports and the various code customizations required to generate them were heavily restricted). We were trying to upgrade our software by 2 major versions and make sure everything was still going to run correctly in spite of the massive upgrade. Each report contained >100 pages of 9 widgets per page and all the data going into these widgets was restricted as well as the calculations and outputs themselves. So we needed to somehow prove that the new system would generate these reports exactly the same as the previous system in spite of the fact that everything had changed under the hood and we couldn’t see either the old or new reports. What we decided to do was treat the entire system as a big blackbox test. I built a junit harness that would generate the pdf of each of a set of reports using the old system and the new system and then diff the output. Initially we literally just used a normal text diff on the pdf file, but once we had fixed the first few (hundred) differences we refined it to snapshot both pdfs to images and produce the diff images because that made it easier to find and fix the problems. It was a very painful process but an extremely effective test because the reports were the end product of the entire system and nailing all the differences in the reports proved that all the input data, all the calculated outputs and everything else was working correctly. It ended up with us doing the upgrade successfully.