3 ms·
> If nothing changes Something changes, but not necessarily something important for every code path. Test could be to run optimization A x 1,000 then optimiza
by Retric 7y ago
> If nothing changes
Something changes, but not necessarily something important for every code path.
Test could be to run optimization A x 1,000 then optimization B x 1,000.
So now I Run the test, edit optimization B, and just run the full test again. Running A again might seem wasteful, but what if something else where to affect A and B, if I just test B then it’s not obvious something other than my code changed. Perhaps as you say the heat sink is failing and my CPU is now down clocking.
The goal is to have a simple test that shows if I am improving things while also isolating that I am measuring what I think I am measuring. That’s not to say I will actually run several long tests repeatedly, but I may re run a test expecting similar results.
Anyway, all of the above has nothing to do with why as want a confidence interval. It’s useful for getting a ballpark feeling of the underlying data.