Well, if they are testing files that are only ever downloaded as part of the test, then all of the CDNs will be on equal footing as far as caching goes (assumin

Well, if they are testing files that are only ever downloaded as part of the test, then all of the CDNs will be on equal footing as far as caching goes (assuming that they randomize the order that CDNs are tested in to try to eliminate biases due to some people quitting before the test is over). Part of what you would be trying to measure is how well the caching infrastructure and strategies actually effect the latency seen by end users. If one CDN keeps things cached on the edge longer, and one aggressively evicts things, that will impact latency which is exactly what you would want to be measuring.