4 ms·
I suspect this is it. Any benchmark that takes less than a second to run should have its iteration count increased such that it takes at least a second, and pre
by drob518 11mo ago
I suspect this is it. Any benchmark that takes less than a second to run should have its iteration count increased such that it takes at least a second, and preferably 5+ seconds, to run. Otherwise CPU scheduling, network processing, etc. is perturbing everything.
- igouy 11mo agoWhat if instead we measured with … BenchExec "uses the cgroups feature of the Linux kernel to correctly handle groups of processes and uses Linux user namespaces to create a container that restricts interference of [each program] with the benchmarking host." https://github.com/sosy-lab/benchexec https://github.com/sosy-lab/benchexec
- drob518 11mo agoCertainly better, but you’re always going to be better off maximizing the runtime to a level where it just swamps any of the other effects. Then do multiple runs and take an average.