3 ms·
10.8.1 improved things a little bit, but running time is still short of where it was in Lion. We only have one Mac in our test suite that can support it, but w
by tanousjm 14y ago
10.8.1 improved things a little bit, but running time is still short of where it was in Lion.
We only have one Mac in our test suite that can support it, but we're going to try to test the 2011 MBP with Snow Leopard and compare that to the results on the same machine for Lion and Mountain Lion.
- antoinevg 14y agoWould also love to see a test of 10.7.0 as a number of us have already been experiencing battery problems since switching from Snow Leopard to Lion that were (partially) sorted out with 10.7.4 (See: https://discussions.apple.com/thread/3194235 https://discussions.apple.com/thread/3194235)
- mikeash 14y agoHow many tests do you run with each configuration? I'm wondering what sort of variation these test have, but it doesn't appear to be discussed.
- tanousjm 14y agoYou're right; I'm sorry I missed discussing that. Our testing methodology is to run each test at least 2 times and if the second test is within 5 percent of the first, to average the two results. If there's a greater than 5 percent difference, we run multiple tests until we can determine which result was abnormal and why. It's not THE most scientific methodology, but it's a decent balance between time and accuracy. In the case of our most recent round of tests, none of the second results were more than 4 percent off the first test.
- mikeash 14y agoI'm really pleased to hear that. Given the lack of discussion I had assumed the worst, but I'm glad to be wrong, and sorry to presume otherwise. Thanks a lot for following up.