I spent much of the Need for Speed sprint on Python doing benchmarking between 2.4 and the 2.5 alpha that we were trying to speed up. It was significantly slow

I spent much of the Need for Speed sprint on Python doing benchmarking between 2.4 and the 2.5 alpha that we were trying to speed up. It was significantly slower than 2.4 at pybench. Measuring performance is hard. It's easy when there are big differences, like the new exceptions in 2.5 were 60% slower. But small things can add up, and it's hard to measure them. One piece of wisdom I got from Tim Peters: Run short benchmarks. The longer you run them the more likely they are to be influenced by context switches and the like. I used to always run long benchmarks hoping to push these changes down into the noise. Instead I started running short tests and looking at the fastest of the runs. Another area of speed improvement though was less about A/B testing and more about thinking hard about where time is used and how to just get rid of it entirely. Some use cases really suffered because in Python strings are immutable. If you are reading from the network and, say, looking for the end of a data frame, you may create a lot of new objects as you append incoming data. IIRC, a "bytearray" was created that was basically a mutable string to get rid of that problem entirely.