I'm not sure about the specific context here, but benchmark warm-ups are often used to make sure that the program's run time is nothing but that, and doesn't in

I'm not sure about the specific context here, but benchmark warm-ups are often used to make sure that the program's run time is nothing but that, and doesn't include things like setting up an environment or compiling to a cache. I think there are sometimes arguments to be made that, for the user, that is run time. When using benchmarks to compare programs like this, though, I think isolating the "actual" (i.e. useful) program execution is productive.