6 ms·
CPython internals: Codewalk through the Python interpreter source code (2014)
- bgilroy26 11y agoI've been hoping to come across something like this ever since Pat Shaughnessy's Ruby Under a Microscope (http://patshaughnessy.net/ruby-under-a-microscope http://patshaughnessy.net/ruby-under-a-microscope). This looks fantastic!
- hyperpape 11y agoLooks interesting. Pedagogically, I wonder what the tradeoffs for using Python are. I haven't looked at it, in part because I've heard comments to the effect that it's a big hairy mess inside. On the upside, students have a better chance of actually needing the knowledge than with Lua, for instance.
- pgbovine 11y ago[author here] it's not too bad, actually; i mean there's the usual hairiness of anything implemented in C, but the interpreter doesn't try to do much in terms of clever optimizations, so the code base is fairly readable for a production-grade system. check it out!
- hyperpape 11y agoI actually have it open, and so far, it's not as much as I thought. Still, the interpreter I've previously looked at is Lua, so it's a lot bigger.
- haberman 11y agoI agree with you that the code is pretty readable, and well-commented. But in my experience with the code-bases of MRI (Ruby), CPython, and Lua, IMO CPython is by far the hairiest. Here is what I mean by "hairy": reading the Python source, you see all sorts of complexity that looks like it is the result of patching problems in previous versions. There are all sorts of comments of the form "we have to do this weird thing here because some obscure things breaks if we don't." The most extreme example of this I've seen is this monster comment, written in the style of a FAQ because it's so complicated: https://hg.python.org/cpython/file/9beeb4c1375d/Objects/typeobject.c#l1249 https://hg.python.org/cpython/file/9beeb4c1375d/Objects/type... I've had so many experiences of wondering "wtf is Python doing, I don't get it," only to find some strange behavior in the interpreter that I never could have anticipated. Things like this won't affect most people writing straight Python, but once you are writing C extensions the inner complexity of CPython becomes far more apparent. Here is an example of a problem I ran into. I had a C extension that was working fine in Python 2.x and 3.3, but then I got bug reports from my users that it was broken in 3.4. It turned out Python 3.4 had added a restriction: it was no longer allowed to make a statically-allocated type derive from a dynamically-allocated type. Why the new restriction? It's not entirely clear: this rule isn't documented anywhere, and the bug/change that added the restriction has no explanation (http://bugs.python.org/issue22079 http://bugs.python.org/issue22079). They probably discovered some scenario where allowing this could lead to problems, so they added some code to disallow it. This is my experience working with CPython. Lots of subtle rules and semantics, many of which are not documented except as comments in the source.
- latenightcoding 11y agoAwesome! I have been trying to find something like this for Lua or Perl
- kbenson 11y agoLike this[1]? 1: http://perldoc.perl.org/perlguts.html http://perldoc.perl.org/perlguts.html
- latenightcoding 11y agoNo, that is a good guide is you are doing XS stuff
- kbenson 11y agoAh, well, maybe one of the pages here is useful: http://perldoc.perl.org/index-internals.html http://perldoc.perl.org/index-internals.html
- sitkack 11y agoP Guo is a Guy Steele level thinker and lecturer. This is good stuff, pay attention.
- ConAntonakos 11y ago100% agree. He writes very incisive articles. He also created the online Python Tutor!
- ahmad19526 11y agoI love this walkthrough! I've been going through it except with the python 3 interpreter.
- enitihas 11y agoI too am interested in doing it with the python 3 interpreter. Can you share the relevant link
- nanoojaboo 11y agoseems like his resume updates automatically, or did he just happen to update it and repost it on the day this link was posted? (October 31, 2015) http://pgbovine.net/philip-guo-cv.pdf http://pgbovine.net/philip-guo-cv.pdf
- saurabhjha 11y agoProfessor Guo writes great articles. His "PhD grind" is great not only for PhD students but also for people in early-stage startups.
- ajcr 11y agoSaw this link on Stack Overflow the other day and have been watching attentively. Great videos - I found the presentation of the PyObject particularly helpful (lecture 4). I'd be overjoyed if there was an extension to the course in future!
- addnab 11y agoIs there something like this similar for Javascript? V8 maybe?