Each of the test cases measured needs to be run at least 3 times in a row, to warm caches (not just CPU but OS too) and to detect and remove noise. In fact, I

Each of the test cases measured needs to be run at least 3 times in a row, to warm caches (not just CPU but OS too) and to detect and remove noise. In fact, I would run the same test repeatedly, keeping track of the k fastest times (k being ~3-7), and only stopping when the first and the kth fastest times are within a certain tolerance (as low as 1%). This ensures repeatability. One sample of performance data for each test is not enough. This study provides no new insights. Performance analyst