3 ms·
I would have to spend a long time studying the relevant Python implementation to think I knew whether there was nondeterminism present in a given benchmark. Tha
by hyperpape 7y ago
I would have to spend a long time studying the relevant Python implementation to think I knew whether there was nondeterminism present in a given benchmark. That's a tall order.
- oefrha 7y agoNow that I think about it, you’re right that randomized hashing in Python itself could cause nondeterminism in seemingly deterministic code. However, timeit does mitigate that to some extent by repeating the same code a large number of times — 1 million times by default, and it’s better used that way. Of course, depending on the actual code, it could be the case that all 1M invocations are skewed by the specific choice of PYTHONHASHSEED; it’s not something that you can solve in-process, so there’s no panacea, you must ditch the idea of getting a single stable number in that case.