why did CPython running the Mandelbrot benchmark have a running time of 134.419s and an incredibly wide confidence interval of ± 32.347s? I had no idea, so, in

why did CPython running the Mandelbrot benchmark have a running time of 134.419s and an incredibly wide confidence interval of ± 32.347s? I had no idea, so, in the best tradition of VM measurements, I buried my head in my sand and hoped that no-one would say anything. They didn’t. A beautiful summary of the field! In other words, this VM is highly optimised for this benchmark — but no-one ever ran it long enough before to notice that performance gets much worse over time. Perhaps, although more likely, they noticed; and then in accordance with tradition, promptly buried their heads in the sands. --- Since this thread is likely to be read by people who care about unexplainable performance glitches, I'll highlight an otherwise inexplicable ~20% run-to-run performance drop that I recently saw an explanation for: https://www.realworldtech.com/forum/?threadid=179700&curpostid=179700 https://www.realworldtech.com/forum/?threadid=179700&curpost... There has been a lot of talk about whether AVX512 is of real world benefit. One big issue is that it's so power hungry that the entire core compensates by slowing down whenever 512-bit instructions are executed. What's not well known is that with an AVX512 capable processor running the common Ubuntu 16.04, you may well encounter this slowdown even if your program never uses AVX512. More specifically, there is a problem with the way that some versions of glibc are saving and restoring the 512-bit ZMM registers. Even at program start-up, the upper 256-bits can be in a "dirty" state, which can cause the core to slow down to the "AVX512-light" speed on any SSE, AVX, AVX2, or FP operation. Patches for glibc were backported as far at 2.25, but Ubuntu hasn't included these fixes in their updates. The bottom line is that if you are working on an AVX512 capable system, you should probably make sure that you understand this bug before publishing any benchmark results.