15 ms·

I don't understand why people care about the CPython GIL. For computationally intense stuff (numerics) running in the interpreter, the language is generally 60

by xscott 5y ago

I don't understand why people care about the CPython GIL. For computationally intense stuff (numerics) running in the interpreter, the language is generally 60 times slower than PyPy and often 100 times slower than an equivalent C program. That means if you want performance, you would need to dodge Amdahl's law and have 60-100 processors for a hypothetical GIL-less CPython to match a single threaded program in PyPy or C.

zmmmmm 5y ago
If all it was used for was scripting it would be fine, but what is strange is that Python has emerged as a major language for complex data analysis. So I am sometimes confronted with partial solutions in Python that solve a piece of the puzzle that is purely numerical (so all runs on numpy etc) but then extending it to actually solve the real problem ends up either extremely awkward (the whole architecture getting bent around Python's limitations) or being a complete rewrite. It's frustrating that the ecosystem for certain problems is so bent towards a language that is unsuitable for large parts of the job. (mainly, I would add, because it sucks the oxygen away from alternatives that would be better in the longer term).
- xscott 5y ago
  > it sucks the oxygen away from alternatives that would be better in the longer term It's always frustrated me that the computing world isn't as it should be. Python as a language seems to have found a sweet spot for accessibility to a lot of people, but CPython the interpreter steals all the air from PyPy, which is a vastly better implementation.
  - seunosewa 5y ago
    PyPy has a GIL too though.
    - xscott 5y ago
      The PyPy FAQ addresses this. To my eyes, it looks like this is partly to provide the same promises CPython does for C extensions, and partly because their transactional memory approach hasn't gotten the attention it would need to succeed. I think both of those show CPython stealing the air from PyPy. https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...
- fastasucan 5y ago
  You cant both claim that it is major language for complex data analysis and that it is "unsuitable for large parts of the job". If it is a major language for complex data analysis it is apparently not unsuitable. It can be not optimal/not the best/bad but it cannot be unsuitable, because then your first statement isn't true. Which one is it?
  - lmm 5y ago
    It's a great language for exploratory prototyping of complex data analysis and a terrible language for robust implementation of complex data analysis. Well-managed organisations prototype in Python but take the time to rewrite the prototype for production use; poorly-managed ones skip step 2.
    - maccard 5y ago
      What language do you rewrite it in so?
      - chalst 5y ago
        It's typically pretty easy to turn Python+numpy into performant Julia.
        rofrol 5y ago
        I remember reading this comment: >I recently tried to migrate my R code to Julia. Even though I already knew R data.table is faster than DataFrames.jl, I was totally blown away by how slow Julia is. So I quickly gave up. I think I will have to write unavoidable hard loop in cpp, which I really don't want to do... https://news.ycombinator.com/item?id=27404910 https://news.ycombinator.com/item?id=27404910
        pjmlp 5y ago
        Apparently it is working for some, https://juliacomputing.com/case-studies/ https://juliacomputing.com/case-studies/
- maccard 5y ago
  If you're doing complex data analysis in python, the actual number crunching isn't happening in python it's happening in a C extension. Using numpy or tensor flow, you're really just calling C code.
  - grumpyprole 5y ago
    And Fortran code (many parts of numpy). C doesn't have a complete monopoly on high-performance code.
    - Qi_ 5y ago
      What is the advantage of Fortran over only C in numpy’s case specifically?
      - staticassertion 5y ago
        Fortran has the more optimized libraries (blas) and is also capable of optimizations that C isn't (guarantees that pointers dont alias).
        qsort 5y ago
        Modern C has the restrict keyword for that. There isn't any competitive advantage left for Fortran over C or C++, the only reason why Fortran is still part of the modern numeric stack is that BLAS, LAPACK, QUADPACK and friends run the freaking world and nobody is ever going to rewrite them to C without a compelling reason to do so.
        staticassertion 5y ago
        I'm aware of the restrict keyword, though its usage is not typical. It enables an optimization that fortran has by default, that was my point. And yes, I already said that packages like blas have had massive amounts of work put into them and have lasting power.
- d0mine 5y ago
  Could you provide a specific example where GIL prevents "complex data analysis"?
Phil_Latio 5y ago
Though it's not only about number crunching. I tried to implement a streaming protocol for certain ip cameras: You may already see the effects of GIL + the general slowness when reading many small packets from a network socket in one thread and parse/process the packets in another. While Python is my favorite tool for many years and I was seemingly aware of it's downsides, I really didn't expect how bad it performed in this case.
- xscott 5y ago
  > [...] when reading many small packets from a network socket [...] Yeah, I should've said it more carefully. That's what I was hinting at with the mention of Amdahl's law. The separate bits of I/O are done in C extensions which can release the GIL, and those can be done in parallel (with each other and your computation), but if each bit of work is small, the synchronization cost (reacquiring the GIL for each packet) becomes relatively large.
OskarS 5y ago
For number crunching it doesn’t matter, but I imagine it’s a much bigger issue if you want to run some kind of application server/webapp on a big beefy machine with lots of cores. That’s a thing you would want to run in something like Python, and the lack of parallelism is a real issue.
- xscott 5y ago
  If your application is I/O bound (web app), then you can get parallelism, even with the GIL. Each I/O call can (should) release the GIL before it waits on it's network packets.
gjulianm 5y ago
I do care about the CPython GIL. I develop a program with a lot of analysis and output to custom formats. The things that can go to the underlying C extension go there, but the development cost is far higher for migrating stuff there. I'd like to be able to parallelize the different sections, but multiprocessing is a mess (only pickable data can be used, no shared state), threading doesn't give me any benefit due to the GIL, and PyPy hasn't worked the last three times I tried. Also, improvements can stack. If I get threading working, I can still move more things to the C extension to improve performance, or migrate finally to PyPy (if they manage to remove the GIL).
- clipradiowallet 5y ago
  > but multiprocessing is a mess Could you share a little more about your experiences with mp? I'm in the other camp - I use multiprocess in several python applications and am more or less happy with it. My real curiosity I suppose is what data you are trying to pass around that isn't a good candidate for mp. My own use cases normally resemble something like multiple worker-style processes consuming their workloads from Queue objects, and emitting their results to another Queue object. Usually my initial process is responsible for consuming and aggregating those results, but in some cases the initial process does nothing but coordinate the worker and aggregator processes.
  - gjulianm 5y ago
    My main issue with multiprocessing is that the pickle module is incredibly unhelpful when something can't be picked. Given that there's quite a lot of context shared between types of analysis, and that context tends to change a lot, using pickle means I'm looking at quite some time debugging every now and then to see what's failing. Regarding the data itself in this case, there are some objects that are references to in-memory structures of a C extension. Last time I checked I didn't see simple ways to share those structures via pickle. Shared memory was an option, but it required me to implement and change quite a lot of things just to get things working, not to mention that whenever the C extension gets new features I need to invest extra time in making those compatible with pickle. It wasn't worth it, specially knowing that I would still run into problems every now and then due to pickling the context.
bno1 5y ago
In ML you often need to stream data from the training set because you can't hold it all in memory. Many times I noticed that my GPU would sit idle because python was busy loading more training examples. With proper multi threading you can have one or multiple threads loading data into a queue feeds the GPU thread. This is something extremely simple to do in any programming language that has doesn't have a GIL. And you can't just use a library for this, loading and preprocesding data is very specific to your application, and writing just this part in C completely defeats the purpose of choosing python in the first place.
kobalsky 5y ago
> I don't understand why people care about the CPython GIL 1. pypy has a GIL 2. not every python program is compatible with pypy. 3. not everyone can program in C. 4. it would be a free speed up for troves of multithreaded code. that being said, the GIL is not going anywhere, there are enough workarounds and the task is monumental.
slaymaker1907 5y ago
In the article, Victor brings up a good point that the GIL can lead to some nasty scheduling with IO. If you have some thread doing computationally expensive stuff and other threads responding to HTTP requests, the computationally intensive thread ends up hogging the CPU. When a thread has been waiting for IO, you often want to switch to it as soon as the IO is finished in order to keep the system responsive. So the biggest issue with the GIL seems to be more about scheduling and less about actually having true concurrent threads in Python.
cma 5y ago
128 thread processors are already available at the prosumer level (hyperthreading is presumably a good use for python since it does so much waiting with pointer chasing, but even if not there will be 128 core consumer chips before long).

by xscott 5y ago

zmmmmm 5y ago

If all it was used for was scripting it would be fine, but what is strange is that Python has emerged as a major language for complex data analysis. So I am sometimes confronted with partial solutions in Python that solve a piece of the puzzle that is purely numerical (so all runs on numpy etc) but then extending it to actually solve the real problem ends up either extremely awkward (the whole architecture getting bent around Python's limitations) or being a complete rewrite. It's frustrating that the ecosystem for certain problems is so bent towards a language that is unsuitable for large parts of the job. (mainly, I would add, because it sucks the oxygen away from alternatives that would be better in the longer term).

xscott 5y ago
> it sucks the oxygen away from alternatives that would be better in the longer term It's always frustrated me that the computing world isn't as it should be. Python as a language seems to have found a sweet spot for accessibility to a lot of people, but CPython the interpreter steals all the air from PyPy, which is a vastly better implementation.
- seunosewa 5y ago
  PyPy has a GIL too though.
  - xscott 5y ago
    The PyPy FAQ addresses this. To my eyes, it looks like this is partly to provide the same promises CPython does for C extensions, and partly because their transactional memory approach hasn't gotten the attention it would need to succeed. I think both of those show CPython stealing the air from PyPy. https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...
fastasucan 5y ago
You cant both claim that it is major language for complex data analysis and that it is "unsuitable for large parts of the job". If it is a major language for complex data analysis it is apparently not unsuitable. It can be not optimal/not the best/bad but it cannot be unsuitable, because then your first statement isn't true. Which one is it?
- lmm 5y ago
  It's a great language for exploratory prototyping of complex data analysis and a terrible language for robust implementation of complex data analysis. Well-managed organisations prototype in Python but take the time to rewrite the prototype for production use; poorly-managed ones skip step 2.
  - maccard 5y ago
    What language do you rewrite it in so?
    - chalst 5y ago
      It's typically pretty easy to turn Python+numpy into performant Julia.
      - rofrol 5y ago
        I remember reading this comment: >I recently tried to migrate my R code to Julia. Even though I already knew R data.table is faster than DataFrames.jl, I was totally blown away by how slow Julia is. So I quickly gave up. I think I will have to write unavoidable hard loop in cpp, which I really don't want to do... https://news.ycombinator.com/item?id=27404910 https://news.ycombinator.com/item?id=27404910
        pjmlp 5y ago
        Apparently it is working for some, https://juliacomputing.com/case-studies/ https://juliacomputing.com/case-studies/
maccard 5y ago
If you're doing complex data analysis in python, the actual number crunching isn't happening in python it's happening in a C extension. Using numpy or tensor flow, you're really just calling C code.
- grumpyprole 5y ago
  And Fortran code (many parts of numpy). C doesn't have a complete monopoly on high-performance code.
  - Qi_ 5y ago
    What is the advantage of Fortran over only C in numpy’s case specifically?
    - staticassertion 5y ago
      Fortran has the more optimized libraries (blas) and is also capable of optimizations that C isn't (guarantees that pointers dont alias).
      - qsort 5y ago
        Modern C has the restrict keyword for that. There isn't any competitive advantage left for Fortran over C or C++, the only reason why Fortran is still part of the modern numeric stack is that BLAS, LAPACK, QUADPACK and friends run the freaking world and nobody is ever going to rewrite them to C without a compelling reason to do so.
        staticassertion 5y ago
        I'm aware of the restrict keyword, though its usage is not typical. It enables an optimization that fortran has by default, that was my point. And yes, I already said that packages like blas have had massive amounts of work put into them and have lasting power.
d0mine 5y ago
Could you provide a specific example where GIL prevents "complex data analysis"?

Phil_Latio 5y ago

Though it's not only about number crunching. I tried to implement a streaming protocol for certain ip cameras: You may already see the effects of GIL + the general slowness when reading many small packets from a network socket in one thread and parse/process the packets in another. While Python is my favorite tool for many years and I was seemingly aware of it's downsides, I really didn't expect how bad it performed in this case.

xscott 5y ago
> [...] when reading many small packets from a network socket [...] Yeah, I should've said it more carefully. That's what I was hinting at with the mention of Amdahl's law. The separate bits of I/O are done in C extensions which can release the GIL, and those can be done in parallel (with each other and your computation), but if each bit of work is small, the synchronization cost (reacquiring the GIL for each packet) becomes relatively large.

OskarS 5y ago

For number crunching it doesn’t matter, but I imagine it’s a much bigger issue if you want to run some kind of application server/webapp on a big beefy machine with lots of cores. That’s a thing you would want to run in something like Python, and the lack of parallelism is a real issue.

xscott 5y ago
If your application is I/O bound (web app), then you can get parallelism, even with the GIL. Each I/O call can (should) release the GIL before it waits on it's network packets.

gjulianm 5y ago

I do care about the CPython GIL. I develop a program with a lot of analysis and output to custom formats. The things that can go to the underlying C extension go there, but the development cost is far higher for migrating stuff there. I'd like to be able to parallelize the different sections, but multiprocessing is a mess (only pickable data can be used, no shared state), threading doesn't give me any benefit due to the GIL, and PyPy hasn't worked the last three times I tried. Also, improvements can stack. If I get threading working, I can still move more things to the C extension to improve performance, or migrate finally to PyPy (if they manage to remove the GIL).

clipradiowallet 5y ago
> but multiprocessing is a mess Could you share a little more about your experiences with mp? I'm in the other camp - I use multiprocess in several python applications and am more or less happy with it. My real curiosity I suppose is what data you are trying to pass around that isn't a good candidate for mp. My own use cases normally resemble something like multiple worker-style processes consuming their workloads from Queue objects, and emitting their results to another Queue object. Usually my initial process is responsible for consuming and aggregating those results, but in some cases the initial process does nothing but coordinate the worker and aggregator processes.
- gjulianm 5y ago
  My main issue with multiprocessing is that the pickle module is incredibly unhelpful when something can't be picked. Given that there's quite a lot of context shared between types of analysis, and that context tends to change a lot, using pickle means I'm looking at quite some time debugging every now and then to see what's failing. Regarding the data itself in this case, there are some objects that are references to in-memory structures of a C extension. Last time I checked I didn't see simple ways to share those structures via pickle. Shared memory was an option, but it required me to implement and change quite a lot of things just to get things working, not to mention that whenever the C extension gets new features I need to invest extra time in making those compatible with pickle. It wasn't worth it, specially knowing that I would still run into problems every now and then due to pickling the context.

bno1 5y ago

In ML you often need to stream data from the training set because you can't hold it all in memory. Many times I noticed that my GPU would sit idle because python was busy loading more training examples. With proper multi threading you can have one or multiple threads loading data into a queue feeds the GPU thread. This is something extremely simple to do in any programming language that has doesn't have a GIL. And you can't just use a library for this, loading and preprocesding data is very specific to your application, and writing just this part in C completely defeats the purpose of choosing python in the first place.

kobalsky 5y ago

> I don't understand why people care about the CPython GIL 1. pypy has a GIL 2. not every python program is compatible with pypy. 3. not everyone can program in C. 4. it would be a free speed up for troves of multithreaded code. that being said, the GIL is not going anywhere, there are enough workarounds and the task is monumental.

slaymaker1907 5y ago

In the article, Victor brings up a good point that the GIL can lead to some nasty scheduling with IO. If you have some thread doing computationally expensive stuff and other threads responding to HTTP requests, the computationally intensive thread ends up hogging the CPU. When a thread has been waiting for IO, you often want to switch to it as soon as the IO is finished in order to keep the system responsive. So the biggest issue with the GIL seems to be more about scheduling and less about actually having true concurrent threads in Python.

cma 5y ago

128 thread processors are already available at the prosumer level (hyperthreading is presumably a good use for python since it does so much waiting with pointer chasing, but even if not there will be 128 core consumer chips before long).