4 ms·

I am surprised the GIL hasn't been mentioned once in the presentation or any of the comments.

windlep 11y ago
Why would it? Using twisted (or py3's asyncio), the event loop is spinning, there is no thread for Python to be switching to. Except that we're using a thread-pool of course (to make external blocking network calls), and those threads are.... making network requests, which is I/O, and the GIL releases during that. On a network bound daemon, which is not CPU-bound, the GIL is really not an issue, so it never came up.
- gtrubetskoy 11y ago
  "On a network bound daemon, which is not CPU-bound" - I think that's a fairly unique constraint which makes your case special. For me, the GIL is unfortunately where Python is unable to compete with Go.
alextgordon 11y ago
node.js has a GIL and nobody talks about that either. The GIL is not relevant because Python is so slow that running it in two threads is hardly an improvement.
- gtrubetskoy 11y ago
  But don't you need to run a bunch of node.js instances in production on a multi-cpu system behind nginx or haproxy? Would it be necessary had it not been for the GIL?
  - alextgordon 11y ago
    Most node.js code spends barely any time on the v8 thread. The node code merely invokes routines in libuv, which has a thread pool for anything computationally intensive.
- trentnelson 11y ago
  I get linear scaling with PyParallel: http://pyparallel.org/ http://pyparallel.org/
- _wmd 11y ago
  This is the weirdest and most nonsensical description of the GIL I've ever seen, and assuredly one coming from a Python/JS non-programmer. In both languages the serialization of runtime is a feature explicitly designed to simplify the programming model and its implementation. There is only one case where the serialized runtime presents a problem, and that is in CPU-bound 90s-style shared memory parallel computation that the industry as a whole has been trying to escape for the past 20 years, because thinking about individual threads and the lifetimes of shared memory allocations turned out to be an incredibly shitty abstraction. Even if you want to shoot yourself in the foot, both Python and Node.js provide facilities to allow e.g. concurrent array access (in Python via the multiprocessing package). The reason those approaches aren't more popular in those languages is exactly because the model itself is defective. Anyone worth their salt working in a computation-heavy domain stopped writing explicit threading code a long time ago.

Why would it? Using twisted (or py3's asyncio), the event loop is spinning, there is no thread for Python to be switching to. Except that we're using a thread-pool of course (to make external blocking network calls), and those threads are.... making network requests, which is I/O, and the GIL releases during that. On a network bound daemon, which is not CPU-bound, the GIL is really not an issue, so it never came up.

gtrubetskoy 11y ago
"On a network bound daemon, which is not CPU-bound" - I think that's a fairly unique constraint which makes your case special. For me, the GIL is unfortunately where Python is unable to compete with Go.

alextgordon 11y ago

node.js has a GIL and nobody talks about that either. The GIL is not relevant because Python is so slow that running it in two threads is hardly an improvement.

gtrubetskoy 11y ago
But don't you need to run a bunch of node.js instances in production on a multi-cpu system behind nginx or haproxy? Would it be necessary had it not been for the GIL?
- alextgordon 11y ago
  Most node.js code spends barely any time on the v8 thread. The node code merely invokes routines in libuv, which has a thread pool for anything computationally intensive.
trentnelson 11y ago
I get linear scaling with PyParallel: http://pyparallel.org/ http://pyparallel.org/
_wmd 11y ago
This is the weirdest and most nonsensical description of the GIL I've ever seen, and assuredly one coming from a Python/JS non-programmer. In both languages the serialization of runtime is a feature explicitly designed to simplify the programming model and its implementation. There is only one case where the serialized runtime presents a problem, and that is in CPU-bound 90s-style shared memory parallel computation that the industry as a whole has been trying to escape for the past 20 years, because thinking about individual threads and the lifetimes of shared memory allocations turned out to be an incredibly shitty abstraction. Even if you want to shoot yourself in the foot, both Python and Node.js provide facilities to allow e.g. concurrent array access (in Python via the multiprocessing package). The reason those approaches aren't more popular in those languages is exactly because the model itself is defective. Anyone worth their salt working in a computation-heavy domain stopped writing explicit threading code a long time ago.