3 ms·
I don't know. When I see that my benchmark has nondeterministic performance, I consider that as a bug and just remove the nondeterminism. For example, if Rust's
by KKKKkkkk1 7y ago
I don't know. When I see that my benchmark has nondeterministic performance, I consider that as a bug and just remove the nondeterminism. For example, if Rust's hashmaps have a random seed, I'd find a way to initialize the hashmap with the same seed every time I run my benchmark. But I guess you can model the performance as a random variable instead, in which case you'll be benchmarking the Hashmap and not your code.
- kardos 7y agoAh but that wouldn't be representative benchmarking. Consider the later example about finding a random word on the large list -- if you fix the random seed and that turns out to search for 'aardvark' then the linear search will perform well, but it won't be a good measure of how fast the program is in more general conditions. Your idea works for comparing machines though.