4 ms·

I'm growing more interested in actually understanding some of the internals that you mention here. I know this is a bit of a tangent, but is there a better way

I'm growing more interested in actually understanding some of the internals that you mention here. I know this is a bit of a tangent, but is there a better way to approach understanding python's internals than reading the source (which feels a bit monolithic to me right now)?

formerly_proven 5y ago
The central dispatch of the VM is a good place to start: https://github.com/python/cpython/blob/main/Python/ceval.c#L1678 https://github.com/python/cpython/blob/main/Python/ceval.c#L...
boothby 5y ago
I thought I had a pretty good handle on Python internals, until some time early this year when I took an interest in the generated bytecode. I'd read plenty of the cpython source, written lots of cython extensions, etc., but somehow missed the middle piece. Fortunately, it's really easy to get at the bytecode, and quite instructive. Random inquiry: how do generator functions work? In [1]: def foo(): ...: yield from range(10) ...: In [2]: import dis In [3]: dis.dis(foo) 2 0 LOAD_GLOBAL 0 (range) 2 LOAD_CONST 1 (10) 4 CALL_FUNCTION 1 6 GET_YIELD_FROM_ITER 8 LOAD_CONST 0 (None) 10 YIELD_FROM 12 POP_TOP 14 LOAD_CONST 16 RETURN_VALUE From there, you can read how each of those bytecode instructions is implemented in ceval.c, which formerly_proven links to. edit: probably nice to have the actual disassembly of a list comprehension, too: In [4]: def bar(): ...: return [x for x in range(10)] ...: In [5]: dis.dis(bar) 2 0 LOAD_CONST 1 (<code object <listcomp> at 0x7f814059f450, file "<ipython-input-4-a7a3cc7e7d7f>", line 2>) 2 LOAD_CONST 2 ('bar.<locals>.<listcomp>') 4 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (range) 8 LOAD_CONST 3 (10) 10 CALL_FUNCTION 1 12 GET_ITER 14 CALL_FUNCTION 1 16 RETURN_VALUE Disassembly of <code object <listcomp> at 0x7f814059f450, file "<ipython-input-4-a7a3cc7e7d7f>", line 2>: 2 0 BUILD_LIST 0 2 LOAD_FAST 0 (.0) >> 4 FOR_ITER 8 (to 14) 6 STORE_FAST 1 (x) 8 LOAD_FAST 1 (x) 10 LIST_APPEND 2 12 JUMP_ABSOLUTE 4 >> 14 RETURN_VALUE

formerly_proven 5y ago

The central dispatch of the VM is a good place to start: https://github.com/python/cpython/blob/main/Python/ceval.c#L1678 https://github.com/python/cpython/blob/main/Python/ceval.c#L...

boothby 5y ago

I thought I had a pretty good handle on Python internals, until some time early this year when I took an interest in the generated bytecode. I'd read plenty of the cpython source, written lots of cython extensions, etc., but somehow missed the middle piece. Fortunately, it's really easy to get at the bytecode, and quite instructive. Random inquiry: how do generator functions work? In [1]: def foo(): ...: yield from range(10) ...: In [2]: import dis In [3]: dis.dis(foo) 2 0 LOAD_GLOBAL 0 (range) 2 LOAD_CONST 1 (10) 4 CALL_FUNCTION 1 6 GET_YIELD_FROM_ITER 8 LOAD_CONST 0 (None) 10 YIELD_FROM 12 POP_TOP 14 LOAD_CONST 16 RETURN_VALUE From there, you can read how each of those bytecode instructions is implemented in ceval.c, which formerly_proven links to. edit: probably nice to have the actual disassembly of a list comprehension, too: In [4]: def bar(): ...: return [x for x in range(10)] ...: In [5]: dis.dis(bar) 2 0 LOAD_CONST 1 (<code object <listcomp> at 0x7f814059f450, file "<ipython-input-4-a7a3cc7e7d7f>", line 2>) 2 LOAD_CONST 2 ('bar.<locals>.<listcomp>') 4 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (range) 8 LOAD_CONST 3 (10) 10 CALL_FUNCTION 1 12 GET_ITER 14 CALL_FUNCTION 1 16 RETURN_VALUE Disassembly of <code object <listcomp> at 0x7f814059f450, file "<ipython-input-4-a7a3cc7e7d7f>", line 2>: 2 0 BUILD_LIST 0 2 LOAD_FAST 0 (.0) >> 4 FOR_ITER 8 (to 14) 6 STORE_FAST 1 (x) 8 LOAD_FAST 1 (x) 10 LIST_APPEND 2 12 JUMP_ABSOLUTE 4 >> 14 RETURN_VALUE