4 ms·

Of course he does not intend to criticize it, but to show what the dumb compiler is. > To be clear: This isn’t to say CPython is bad, or even that it should ne

Of course he does not intend to criticize it, but to show what the dumb compiler is. > To be clear: This isn’t to say CPython is bad, or even that it should necessarily change. In fact, as I’ll show, dumb bytecode compilers are par for the course. In the past I’ve lamented how the Emacs Lisp compiler could do a better job, but CPython and Lua are operating at the same level. There are benefits to a dumb and straightforward bytecode compiler: the compiler itself is simpler, easier to maintain, and more amenable to modification (e.g. as Python continues to evolve). It’s also easier to debug Python (pdb) because it’s such a close match to the source listing.

hermitdev 8y ago
Right, the CPython runtime is first trying to be correct (IMHO is also correct). Very few optimizations are made by the compiler/interpreter. This has long been known by Python devs, and exploited by specifically using APIs known to be written in C/C++. e.g. using 'map' instead of a list comprehension. Map was generally faster (at least in the 2.x days) than an equivalent list comprehension.
- joshuamorton 8y ago
  Just FYI this specific wisdom you're suggesting is exactly backwards, or at least could be depending how how you read it. timeit.timeit(setup='x = range(10000); l = lambda n: n + 1', stmt='map(l, x)', number=1000) provides approximately 1.2 seconds to do that. The same with a listcomp: timeit.timeit(setup='x = range(10000)', stmt='[n+1 for n in x]', number=1000) runs in a half second. Although dropping the inlined function runs slower still (1.6s): timeit.timeit(setup='x = range(10000); l = lambda n: n + 1', stmt='[l(n) for n in x]', number=1000) The second is the fastest because you drop the overhead of the function call, which does stack pushes and pops, and instead do them locally. The last is the slowest because it does the stack pushes and pops, as well as loads extra globals, which is slow. On 3.6 the map version runs in 0.00067 seconds, because it doesn't actually do anything, it just constructs a generator object. The general point is well taken. My favorite bit of trivia like this is the fast matrix transpose: `zip(*matrix)`, which does everything in pure c, and is also likely the shortest way to do the transpose in python.
  - fiddlerwoaroof 8y ago
    zip(*matrix) is the coolest pythonism.
  - masklinn 8y ago
    > Just FYI this specific wisdom you're suggesting is exactly backwards, or at least could be depending how how you read it. Yeah. map is usually faster if you already have a function to invoke (especially if that function is a builtin or in C). If you have an expression and have to wrap it in a lambda to get it in map, it's going to be way slower than the listcomp because all things considered CPython's function calls are expensive.
ChrisSD 8y ago
Aye, I did not think he was criticising. However on first read it did sound to me like he was writing as though he was uncovering some arcane fact.

hermitdev 8y ago

Right, the CPython runtime is first trying to be correct (IMHO is also correct). Very few optimizations are made by the compiler/interpreter. This has long been known by Python devs, and exploited by specifically using APIs known to be written in C/C++. e.g. using 'map' instead of a list comprehension. Map was generally faster (at least in the 2.x days) than an equivalent list comprehension.

joshuamorton 8y ago
Just FYI this specific wisdom you're suggesting is exactly backwards, or at least could be depending how how you read it. timeit.timeit(setup='x = range(10000); l = lambda n: n + 1', stmt='map(l, x)', number=1000) provides approximately 1.2 seconds to do that. The same with a listcomp: timeit.timeit(setup='x = range(10000)', stmt='[n+1 for n in x]', number=1000) runs in a half second. Although dropping the inlined function runs slower still (1.6s): timeit.timeit(setup='x = range(10000); l = lambda n: n + 1', stmt='[l(n) for n in x]', number=1000) The second is the fastest because you drop the overhead of the function call, which does stack pushes and pops, and instead do them locally. The last is the slowest because it does the stack pushes and pops, as well as loads extra globals, which is slow. On 3.6 the map version runs in 0.00067 seconds, because it doesn't actually do anything, it just constructs a generator object. The general point is well taken. My favorite bit of trivia like this is the fast matrix transpose: `zip(*matrix)`, which does everything in pure c, and is also likely the shortest way to do the transpose in python.
- fiddlerwoaroof 8y ago
  zip(*matrix) is the coolest pythonism.
- masklinn 8y ago
  > Just FYI this specific wisdom you're suggesting is exactly backwards, or at least could be depending how how you read it. Yeah. map is usually faster if you already have a function to invoke (especially if that function is a builtin or in C). If you have an expression and have to wrap it in a lambda to get it in map, it's going to be way slower than the listcomp because all things considered CPython's function calls are expensive.

ChrisSD 8y ago

Aye, I did not think he was criticising. However on first read it did sound to me like he was writing as though he was uncovering some arcane fact.