4 ms·
It's probably down to the measurement noise of benchmarking on GitHub actions.
by AlotOfReading 10mo ago
It's probably down to the measurement noise of benchmarking on GitHub actions.
- drob518 10mo agoI suspect this is it. Any benchmark that takes less than a second to run should have its iteration count increased such that it takes at least a second, and preferably 5+ seconds, to run. Otherwise CPU scheduling, network processing, etc. is perturbing everything.
- igouy 10mo agoWhat if instead we measured with … BenchExec "uses the cgroups feature of the Linux kernel to correctly handle groups of processes and uses Linux user namespaces to create a container that restricts interference of [each program] with the benchmarking host." https://github.com/sosy-lab/benchexec https://github.com/sosy-lab/benchexec
- drob518 10mo agoCertainly better, but you’re always going to be better off maximizing the runtime to a level where it just swamps any of the other effects. Then do multiple runs and take an average.