7 ms·
How the CPython compiler works
- r4victor 6y agoHi! This is the second part of my new Python behind the scenes series that covers the compilation of a Python program. I'd be glad to hear your thoughts on this part and the series in general.
- Ennea 6y agoThis could be an interesting read, but so far I've only had time to skim it. What I did notice, however, is that some of your images are too wide and get cut off :)
- r4victor 6y agoI know the site is not well adapted for mobiles. I currently use default Pelican theme, which is not responsive, and have plans to change it. Nevertheless, I have no problems with displaying images on any device (desktop/ipad/iphone), so could you elaborate on this? What device do you use? You can email me: victor@tenthousandmeters.com
- Ennea 6y agoDesktop, actually. Full HD. The width of the content is only 800px, but some of your images (like diagram2.png) is 1044px wide. And instead of the image being scaled, it is just being cut off. Edit: Oh yea, there we go. You're using CSS zoom[0], which is non-standard. [0] https://developer.mozilla.org/en-US/docs/Web/CSS/zoom https://developer.mozilla.org/en-US/docs/Web/CSS/zoom
- lordgrenville 6y agoSame for me. Firefox desktop on macOS https://pasteboard.co/Jsysfju.png https://pasteboard.co/Jsysfju.png
- r4victor 6y agoThanks for reporting! Fixed this. I used a non-standard `zoom` attribute, which works in Chrome and Safari. It's a good habit to check how the website looks in all major browsers.
- tasubotadas 6y agoCould you tell us why CPython still doesn't have JIT? What's so hard about it (on Python)?
- creata 6y agoWhy should CPython have a JIT? A good JIT is hard to write, and harder to retrofit. It usually increases startup times, attack surface, and code complexity. Besides, Python already has a handful of JIT-based implementations, and I don't see how adding the reference implementation (which out to be simple!) to that list really improves anything.
- tasubotadas 6y agoYou wrote a lot but didn't contribute to answering the question.
- creata 6y ago1. That's not a lot. 2. It's a forum, not everything has to be a direct response. 3. Either way, this was a direct response: CPython does not have a JIT because there's little reason for it to have a JIT, and many good reasons why it shouldn't. 4. If you want a JIT'd Python so badly, why not use PyPy?
- Rochus 6y agoThere is already https://www.pypy.org/ https://www.pypy.org/ which is about four times faster than CPython (which is not that much actually compared to e.g. V8, see e.g. https://benchmarksgame-team.pages.debian.net/benchmarksgame/which-programs-are-fastest.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...)
- glandium 6y agoCaveat: pypy is not _consistently_ faster than CPython. There are cases where it is slower, sometimes even by a large margin.
- deleted 6y ago[deleted]
- Rochus 6y agoInteresting, looking forward reading it. I recently bought "Inside The Python Virtual Machine" (see https://leanpub.com/insidethepythonvirtualmachine https://leanpub.com/insidethepythonvirtualmachine) which is also recommendable.
- r4victor 6y agoSecond this book. I've also made a list of good resources on CPython internals: https://tenthousandmeters.com/materials/python-behind-the-scenes-a-list-of-resources/ https://tenthousandmeters.com/materials/python-behind-the-sc...
- xaedes 6y agoI once was interested what exactly happens when executing python code, asking myself questions like "Why can't I make a fast for loop? Where are the fast integers? Can't I write fast for loop directly in bytecode, akin to asm?" The python bytecode interpreter mainloop can be found in ceval.c. There is a big switch case for each python bytecode opcode starting at https://github.com/python/cpython/blob/master/Python/ceval.c#L1480 https://github.com/python/cpython/blob/master/Python/ceval.c... One can see even the simplest operations call quite a lot of stuff. I gave up on fast pure cpython after reading ceval.c...
- sigwinch28 6y agoI recall from my days hacking away on the BEAM (the de facto Erlang VM) that it loads code as directly threaded to improve performance. The BEAM loads whole modules at a time, and as part of this process it means that instruction names in the bytecode are essentially replaced with the address of the code which interpret that instruction. This means that once loading is complete there's no big switch statement to visit: the VM jumps directly to the part of its own code which interprets that instruction.
- sigwinch28 6y agoOh wow, my brain was cooked when I wrote that. The BEAM loads bytecode into memory and then goes through the loaded code to directly thread it. It's not an inherent consequence of code loading in general. https://en.wikipedia.org/wiki/Threaded_code#Direct_threading https://en.wikipedia.org/wiki/Threaded_code#Direct_threading
- devxpy 6y agoYou should look at numba. In some special cases, you can actually write for loops in pure python that are faster than C loops.
- JesseMeyer 6y agoAre you referring to Numba's ability to offload certain loops into a GPU kernel? Otherwise, Clang has access to the same optimizations that Numba has as they both share LLVM as their optimizing compiler. Beyond that, I think a fairer comparison is C w/ OpenMP vs Numba for parallel processing if syntactic brevity is the metric.
- miguendes 6y agoGreat stuff. I'm very interested in knowing more about python's internals. I recently brought Anthony Shawn's Cpython internals book but haven't read it yet. [1] [1] https://realpython.com/products/cpython-internals-book/ https://realpython.com/products/cpython-internals-book/