20 ms·

The GIL and its effects on Python multithreading

221 points by WoodenChair 5y ago

TekMol 5y ago
The GIL allows only one OS thread to execute Python bytecode at any given time Is this real? So on top of being 6x slower than PHP and Javascript, Python does not benefit from more than one processor core? So in a web server environment with - say - 4 cores, it will be 24 times slower?
- Epa095 5y ago
  The devil is in the detail, especially the "execute python byte code" part, and how often that happens. Partiluarly, it does not happen when you are waiting for some io, or if you are inside some c library like numpy. But yeah, python is not fast, https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ have quite a lot of benchmarks for those interested.
- zozbot234 5y ago
  Aren't PHP and JS also single-threaded in the exact same way? Regardless, if you look at the history of Python it was supposed to be a teaching language and to be used for very simple, user-written scripts. Using it for actual production code makes about as much sense as writing production stuff in bash, you can do it but you're just making things harder on yourself for no good reason. Even Ruby is a bit better, but these days there's no reason not to use Go or Rust.
  - TekMol 5y ago
    TFA makes it sound like no two Python programs can execute simultaneously on two different cores. I would be very, very surprised if that would hold true for PHP or JS. I have the feeling what the TFA really means is something different. But I cannot really figure out what.
    - Ndymium 5y ago
      What the article means is that a single Python OS process can have multiple OS threads but only one of them can execute Python bytecode at any given time. So if you have a multithreaded Python application (on CPython), only one of those threads will run your code at a time while the others are waiting. Naturally you can have multiple OS processes of Python that run in parallel. This is how Python web services usually work. Then, of course, you need to share memory explicitly via some mechanism like Redis or SysV shared memory, so it leads to different design considerations.
    - ynik 5y ago
      You can only have a single core running Python code per process. You can also have a single Python process running multiple threads in parallel, as long as all but one of those threads are currently running C code (numpy, system calls, ...) But if your program spends more than a tiny fraction of time in interpreted Python code, the GIL will slow you done if you try to use all cores. You can have multiple processes running Python at the same time -- and indeed, many Python programs work around the GIL by forking sub-processes. This tends to introduce significant extra complexity into the program (and extra memory usage, as the easiest solution is typically to copy data to all processes).
      - Symmetry 5y ago
        If your inputs are immutable and you don't want to copy them that works too since if you're careful. Copy on write means that it won't be duplicated by default. Of course reference counting changes means you will be writing to the inputs by default but if you're just branching for a computation you can temporarily turn off garbage collection while you diverge. Disabling garbage collection for a short lived forked process can even make sense on its own terms.
    - masklinn 5y ago
      > TFA makes it sound like no two Python programs can execute simultaneously on two different cores It does not. It states that two python threads can’t execute bytecode simultanously, the implication is that that’s within a single process (which is the case) otherwise talking about threads makes no sense. > I have the feeling what the TFA really means is something different. But I cannot really figure out what. TFA means exactly what it says. Two threads can’t execute python bytecode in parallel, because there’s a big lock around the main interpreter loop (that is in essence what the GIL is). That is quite obviously within the confines of a single Python interpreter = runtime = process.
  - funklute 5y ago
    > Using it for actual production code makes about as much sense as writing production stuff in bash This is a really strange perspective, given how many shops use python for production these days. > Ruby is a bit better, but these days there's no reason not to use Go or Rust. Unless you're specifically looking to access for example machine-learning resources, that are not nearly as mature in those other languages.
  - lmm 5y ago
    > if you look at the history of Python it was supposed to be a teaching language No it wasn't. It was supposed to be a language for automating sysadmin scripts, or for embedding in a program to run user-scripts.
  - tored 5y ago
    PHP can be built as either thread safe (TS (the term ZTS, Zend Thread Safe, is also used)) or non thread safe (NTS), depending how you deploy PHP (how the webserver interacts with PHP like FastCGI or SAPI, if it spawns processes or threads) you run one or the other, most common is that you use NTS builds on Linux and TS builds on Windows. PHP does not have a GIL, thus PHP does not suffer from any slowdown when using threads, proper use of threading improves PHP performance without the need to escape to a C library. Internally PHP uses something called Thread-Safe Resource Manager (TSRM) when built as thread safe. TSRM architecture was improved with PHP 7 and to my understanding performance as well. However PHP does not ship with any API to handle threads, this is because normally you, the web programmer, don't do any process/thread handling in a web setup, concurrency is handled by the webserver for each request. It does exist third party extensions that enables threading API, like in a cron job.
  - joconde 5y ago
    > Using it for actual production code makes about as much sense as writing production stuff in bash, you can do it but you're just making things harder on yourself for no good reason. Even Ruby is a bit better, but these days there's no reason not to use Go or Rust Good luck doing any deep learning in Go or Rust :)
    - pjmlp 5y ago
      No problem with the "Python" libraries that are actually C, C++ or Fortran code.
- CrLf 5y ago
  If you are CPU-bound in pure Python code you’ve already lost. However, most of the time you’re doing CPU-heavy tasks in Python you’re likely using libraries that are ultimately implemented in another language for performance. The GIL can be (and is) released in those cases. Not much of a problem there. It’s also released when you’re waiting for I/O. Not much of a problem there either. Fear of the GIL have never stopped me from using threading in Python, and it has never been much of a problem. In the rare occasions I’ve noticed, I was doing something wrong, not using the appropriate library, or both. Or it might be the case that Python was just a poor choice for the problem, no matter how flexible it is. I also wonder what would happen if CPython got rid of the GIL. Python has certain characteristics that make it hard to have a parallel interpreter, and people have tried (and failed) to remove te GIL before. Don’t fear the GIL.
- BiteCode_dev 5y ago
  This is for real, but this doesn't mean you can't use parallelism. Only that you can't use thread for them most of the time . Parallelism strategies in Python usually revolve around using the multprocessing module, a c extension (numpy will release the gil, and hence all the science stack) or using a 3rd party tools (dask is pretty popular, and allow easy parallelism: https://dask.org/ https://dask.org/).
  - bobbylarrybobby 5y ago
    Even python's parallelism is subpar because objects passed across processes have to be pickled. This both takes time and limits the kinds of objects that can be sent across processes (no lambdas, for instance).
- fastball 5y ago
  No, because in a webserver environment where you have 4 cores, you should be running 4 instances of your program, load-balanced behind nginx or caddy or w/e. Also what's your source for the claim that Python is 6x slower than JS or PHP?
- quietbritishjim 5y ago
  > The GIL allows only one OS thread to execute Python bytecode at any given time [within each OS process] Different Python processes can certainly use different CPU cores - the sentence is definitely misleading. (As others have mentioned, it also only applies to interpreting Python byte code itself, not OS calls or C extensions modules called from that bytecode.)
- jrochkind1 5y ago
  Ruby has the same issue. But a "web server environment" scales pretty well "horizontally", meaning in ruby the solution/workaround is generally to run 4 separate web worker processes if you have 4 cores, so they won't be limited by the shared GIL. I am a rubyist and don't know much about python, but I'd assume python web server environments do the same? Is this as good as a high-performance concurrency platform (like, say, the JVM has become)? Will it lead to absolutely optimal utilization of all cores? No. But the result is not simply "24 times slower", and all cores will be in use. Performance numberes like "6x slower" are almost always contextual when it comes to the real world, it's not like every single program written in python takes 6x longer/6x as much CPU as the nearest-equivalent program in PHP, that's not how it works, it will depend on what the program is doing and how it's deployed.
- acdha 5y ago
  The problem is that performance is variable. For example, it's not just wrong but meaningless to say "6x slower” because there are cases where Python is either faster or slower than PHP or JavaScript and you need to know what specific things you're comparing it to provide a number. For example, I've seen people write entire programs in C and, after taking several weeks, realize their code was nowhere near as fast as the Python version because the code was bottlenecked on dictionary operations and Python's stdlib version was far more tuned than the C implementation they'd written (repeat for image handling where they wrote their own decoder, etc.). This is not to say that a C programmer shouldn't be able to beat Python — only that performance is not a boolean trait and it's important to know where the actual work is happening. The important thing to know is that the GIL can be and is commonly released by C extensions. If your programs tend to be I/O bound (e.g. many web applications) or are using the many extensions which release the GIL as soon as they start working, your performance characteristics can be completely different from someone else's program which is CPU-bound and has different thread interactions. This is why you can find some people who think the GIL is a huge problem and others who've only rarely encountered it. These days, I would typically move really CPU-bound code into Rust but I would also note that in the web-ish space I've worked in the accuracy rate of someone blaming the GIL for a performance issue has been really low[1] compared to the number of times it was something specific to their algorithm. 1. I'm reminded of the time someone went on a rant about how Python was too slow to use for a multithreaded program working with with gzipped-JSON files — 3 months and an order of magnitude more lines of Go later they had less than a 10% win because Python's zlib extension drops the GIL when the C code starts running and was about the same speed as Go's gzip implementation, both running much faster than storage. The actual problem was that the Python code was originally opening and closing file handles on a network filesystem for every operation instead of caching them.
fastball 5y ago
For anyone looking for something on the same topic but a little bit lighter / more practical, I really enjoyed this talk[1] by Raymond Hettinger at PyBay 2017, where he discussed the GIL as well. [1] https://www.youtube.com/watch?v=9zinZmE3Ogk https://www.youtube.com/watch?v=9zinZmE3Ogk
TheChaplain 5y ago
As an avid user of Python I can calm any readers that the GIL issue will not have any effect on a considerable majority of your use cases. Python is not broken but works very well and helps people make millions all over the world.
- matsemann 5y ago
  I've been bitten by the GIL multiple times since switching to a company writing mostly python. Not that I write that much multithreaded code, but the GIL influences everything so lots of patterns in other languages no longer work. For instance, a local cache of some values is basically useless when python is used as backend. Since there will be multiple processes handling the requests (in the same node), they don't share memory. So basically everything has to be stashed at redis. Of course, with scale and multiple nodes lots of stuff should be cached in redis anyway. But not everything makes sense to reach out to an external service for. Like fetching a token I need to use multiple times, instead if having it stored locally in a variable. There are of course ways around this, using some locally shared file or mmap or whatever, but it's a hassle compared to just setting a variable. Same with background tasks or spinning up extra threads to handle the workload of an incoming request. Can't easily be done, so basically every python project includes celery or something similar. Which works fine, but again, a different and forced way to solve something. Asyncio solves many of the daily problems I would have used threads for, though. As I often used it other places to fire off some http requests to other services in parallel. But unfortunately async doesn't work properly with everything yet. So everything has a solution, but you are forced to do it "the python way" and with some hurdles.
  - pjmlp 5y ago
    What about doing it the UNIX way before pthreads became a thing?
    - gpderetta 5y ago
      Well, yes, but there is a reason why pthreads became a thing.
deleted 5y ago
[deleted]
happy_path 5y ago
Weren't a Python project called interpreters that had the aim of allowing different interpreters of Python, thus mitigating the the issues the GIL causes?
- BiteCode_dev 5y ago
  Yes, a first version has been released, but it still needs a lot of ironing.
  - happy_path 5y ago
    Awesome! Would it be ready for the next version?
    - BiteCode_dev 5y ago
      Unlikely. The module is usable, and you can already import it and run it. But the perf benefit are still not there.
      - germandiago 5y ago
        What is this module about actually? I mean, is it a model to run code in parallel in some way different to multiprocessing?
        BiteCode_dev 5y ago
        For now it's only at the c level, with a python api in discussion: https://www.python.org/dev/peps/pep-0554/ https://www.python.org/dev/peps/pep-0554/ The idea is that the python code interpreter would not rely on global variables anymore, so that you can instantiate it several times. The GIL is tied to the interpreter, meaning 2 threads in 2 different interpreters do not share a gil, and can be schedule on different CPU.
- frakkingcylons 5y ago
  That project is mentioned in the article.
est 5y ago
tl;dr don't use threading in CPython unless you know what you are doing. In fact projects like uWSGI disables _thread.so by default.
- BiteCode_dev 5y ago
  Rather, don't use threading for the sake of parallelism in Python. Threads in Python are mostly useful to avoid blocking: - you need to make io calls and don't want to bring asyncio - you need to process things in the background without blocking the main thread (e.g: zippping things while you use a GUI) - you want to run a long process on a c-extension based data structure (they release the gil, so you get the perf boots), such as a numpy array or a pandas DF
___luigi 5y ago
Is there any reference to delve in the details of GIL, multithreading, multiprocessing, high performance computing on top of Python eco-system?. There are some interesting projects such nimba, Dask, etc, but the resources are a bit scattered.
xscott 5y ago
I don't understand why people care about the CPython GIL. For computationally intense stuff (numerics) running in the interpreter, the language is generally 60 times slower than PyPy and often 100 times slower than an equivalent C program. That means if you want performance, you would need to dodge Amdahl's law and have 60-100 processors for a hypothetical GIL-less CPython to match a single threaded program in PyPy or C.
- zmmmmm 5y ago
  If all it was used for was scripting it would be fine, but what is strange is that Python has emerged as a major language for complex data analysis. So I am sometimes confronted with partial solutions in Python that solve a piece of the puzzle that is purely numerical (so all runs on numpy etc) but then extending it to actually solve the real problem ends up either extremely awkward (the whole architecture getting bent around Python's limitations) or being a complete rewrite. It's frustrating that the ecosystem for certain problems is so bent towards a language that is unsuitable for large parts of the job. (mainly, I would add, because it sucks the oxygen away from alternatives that would be better in the longer term).
  - xscott 5y ago
    > it sucks the oxygen away from alternatives that would be better in the longer term It's always frustrated me that the computing world isn't as it should be. Python as a language seems to have found a sweet spot for accessibility to a lot of people, but CPython the interpreter steals all the air from PyPy, which is a vastly better implementation.
    - seunosewa 5y ago
      PyPy has a GIL too though.
      - xscott 5y ago
        The PyPy FAQ addresses this. To my eyes, it looks like this is partly to provide the same promises CPython does for C extensions, and partly because their transactional memory approach hasn't gotten the attention it would need to succeed. I think both of those show CPython stealing the air from PyPy. https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...
r4victor 5y ago
Hi! I'm the author of this article. Thanks for posting it. The GIL is an old topic, but I was surprised to learn recently that it's much more nuanced than I thought. So, here is my little research. This article is a part of my series that dives deep into various aspects of the Python language and the CPython interpreter. The topics include: the VM; the compiler; the implementation of built-in types; the import system; async/await. Check out the series if you liked this post: https://tenthousandmeters.com/tag/python-behind-the-scenes https://tenthousandmeters.com/tag/python-behind-the-scenes I welcome your feedback and questions, thanks!
- sirfz 5y ago
  Haven't checked the full series yet but this GIL article is very nicely written and insightful. Thank you
- jeroenvlek 5y ago
  You filled in a lot of gaps in my GIL knowledge. Today I learned something new. Thank you for writing it!
- gadrev 5y ago
  Wow, just skimmed over a few articles in the series and they seem very nice and well researched. Congratulations.
- harpiaharpyja 5y ago
  In the opening paragraph you state that the GIL prevents speeding up CPU-intensive code by distributing the work among multiple threads. My understanding is that distributing work across multiple threads would not speed up CPU-intensive code anyways. In fact it would add overhead due to threading. Perhaps you meant I/O bound code?
  - jjice 5y ago
    If it was being distributed amongst python threads (which run on one hardware thread), then CPU performance can't improve since they're just taking turns using the CPU. If you're running on multiple hardware threads (what I assume the author meant), that can causes better CPU performance since it will distribute work across real threads that can run in parallel. The GIL restricts use of multiple hardware threads.
helpfulmandrill 5y ago
Ruby seems to have got past its GIL problems with Ractor[1]. What's stopping Python from doing something similar? [1] https://scoutapm.com/blog/ruby-ractor https://scoutapm.com/blog/ruby-ractor
- r4victor 5y ago
  As far as I understand, Ruby's Ractors are what Python is now trying to do with subinterpreters (https://lwn.net/Articles/820424/ https://lwn.net/Articles/820424/). I mention them in the post.
  - danachow 5y ago
    I just skimmed through your link. Subinterpreters have been in the C api for a long time and never got around the GIL, and reading over the post that you linked — nothing has really changed.
    - r4victor 5y ago
      This is a WIP. Here's how it's going on: https://pythondev.readthedocs.io/subinterpreters.html https://pythondev.readthedocs.io/subinterpreters.html
- georgyo 5y ago
  Python has multiprocessing which also bypasses the GIL. It also works with message passing.
  - helpfulmandrill 5y ago
    Much more heavyweight than Ractors though, right?
  - mzs 5y ago
    Which was broken on macOS until 3.8 and is fragile in the face of threading and freezing.
- 5y ago
jrochkind1 5y ago
> Literally, there is an initiative to introduce multiple GILs to CPython. It's called subinterpreters. The idea is to have multiple interpreters within the same process. Threads within one interpreter still share the GIL, but multiple interpreters can run parallel. No GIL is needed to synchronize interpreters because they have no common global state and do not share Python objects. This sounds very much like Ruby's recently introduced "ractor", no?
aynyc 5y ago
For those who find GIL impacts their developments, can you post some examples? I'm curious as to what they are.
- jillesvangurp 5y ago
  Well many developers end up using different languages for this reason. E.g. Go and node.js are popular and both do asynchronous stuff effortlessly (even though they are single threaded typically). And if you use Jython (python on the jvm), you can actually do threads properly as the Gil is not a thing there: https://stackoverflow.com/questions/1120354/does-jython-have-the-gil https://stackoverflow.com/questions/1120354/does-jython-have... Likewise, Iron Python also has no Gil apparently. Both are running quite a bit behind PythonC at this point unfortunately. For properly multi threaded stuff, people use more higher level languages like Erlang, Scala, Java, Kotlin, Rust, C++, etc. In general batch processing data is both IO and CPU intensive (e.g. parsing, regular expressions) and a common use-case for Python. You could say, data science is actually the primary use case driving its popularity. Python is effectively single threaded for that stuff. Using multiple processes fixes the problem well enough though. Conveniently the module for using processes looks very similar to the one for using threads, so switching is easy once you figure out why shit doesn't scale at all (been there done that). But using processes also adds some complexity. The good thing is that it makes doing why the Gil is needed (sharing data between threads/processes) a lot harder/impossible. The bad news is that it's hard. Typically people use queues and databases for this. ETL systems like Airflow tend to use multiple processes and queues instead of threads as using python threads is kind of pointlessly futile (from a making things faster point of view). However, with Airflow a pattern that you see a lot is that python process workers are typically not used and instead it is configured to use one of several cloud based schedulers (e.g. kubernetes, ECS, etc) to schedule work. Basically, airflow is just used to orchestrates work where the actual work happens elsewhere and may or may not involve python code. A lot of the performance critical stuff is native anyway.
  - doctordesh 5y ago
    > ... Go and node.js are popular and both do asynchronous stuff effortlessly (even though they are single threaded typically) I know that I'm being very pedantic here, but Go is not a "typically single threaded" language. Go has as fine control over it's threads (goroutines) as C does over it's threads. Go is very capable of true parallellism.
Mikeb85 5y ago
Honestly, not sure why people care so much about the GIL in languages like Ruby, Python or JS. Let's face it, nowadays when you deploy software it's over multiple machines, nodes, whatever you want to call it. So you're almost always talking about spawning processes. Which any language can do, GIL or not. Removing the GIL to enable parallel processing on a single machine adds more complications than it's worth, especially when any Python workload is either going to be forking C processes or forking Python processes. Not every language needs the same kind of control over OS threads of a C, C++ or Rust.
- zamadatix 5y ago
  People don't care as much about removing the GIL in JS because JS never exposed threads. When the time came for spreading compute load to multiple cores JS decided to simply create multiple contexts first with just message passing and then later with shared memory. As a result you can still code JS in the typical thread unaware way but utilize multiple cores without having to go multi-process or multi-machine. For in thread stuff the async model (or historically callbacks) is good enough if you're not compute bound. In Python threads are exposed but only one runs at a time and you have to go full multiprocessing to use the other cores. This is both confusing and annoying respectively hence the increased noise about the GIL.
- cma 5y ago
  We already have 64 core/128 thread prosumer machines and it is only going to be more in the future. Python is so slow that message passing between processes can often be slower than just handling it in the same process. What if you have a big array of python objects that you want to sort? In other languages it is trivial to get a huge speedup with more cores.
cmacleod4 5y ago
The multiple interpreters communicating by message passing model mentioned towards the end of this article as a way forward is exactly the threading model which Tcl has been using successfully for the last 20 years. For more info see https://www.activestate.com/blog/threads-done-right-tcl/ https://www.activestate.com/blog/threads-done-right-tcl/ and https://www.tcl.tk/man/tcl/ThreadCmd/thread.html#M44 https://www.tcl.tk/man/tcl/ThreadCmd/thread.html#M44 .
- celeritascelery 5y ago
  Are tcl functions shared between threads? Or does each new thread interpreter start from a clean state?
  - cmacleod4 5y ago
    Functions (procs) and other state are not shared between interpreters. So they do need to be reinitialised in each thread. This is not hard to do though. The messages which can be passed between threads can be arbitrary scripts to be executed in the target interpreter, such as proc definitions.
  - blacksqr 5y ago
    The Tcl thread library allows you to create multiple threads in a single interpreter, and use mutexes. locks etc. to manage them. In this case of course you only need to initialize your interpreter once. The library is most used in an "easy" mode however, where each thread starts up with its own interpreter and thus the usual mutex/lock dance is unnecessary. Various forms of message passing can be done among these interpreters, including tools provided by the thread library itself, Tcl's built-in synthetic channel feature which makes communication look like standard file I/O, or, since each interpreter has its own event loop, Tcl's socket I/O features can be used to set up server/client comms.

221 points by WoodenChair 5y ago

The GIL allows only one OS thread to execute Python bytecode at any given time Is this real? So on top of being 6x slower than PHP and Javascript, Python does not benefit from more than one processor core? So in a web server environment with - say - 4 cores, it will be 24 times slower?

Epa095 5y ago
The devil is in the detail, especially the "execute python byte code" part, and how often that happens. Partiluarly, it does not happen when you are waiting for some io, or if you are inside some c library like numpy. But yeah, python is not fast, https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ have quite a lot of benchmarks for those interested.
zozbot234 5y ago
Aren't PHP and JS also single-threaded in the exact same way? Regardless, if you look at the history of Python it was supposed to be a teaching language and to be used for very simple, user-written scripts. Using it for actual production code makes about as much sense as writing production stuff in bash, you can do it but you're just making things harder on yourself for no good reason. Even Ruby is a bit better, but these days there's no reason not to use Go or Rust.
- TekMol 5y ago
  TFA makes it sound like no two Python programs can execute simultaneously on two different cores. I would be very, very surprised if that would hold true for PHP or JS. I have the feeling what the TFA really means is something different. But I cannot really figure out what.
  - Ndymium 5y ago
    What the article means is that a single Python OS process can have multiple OS threads but only one of them can execute Python bytecode at any given time. So if you have a multithreaded Python application (on CPython), only one of those threads will run your code at a time while the others are waiting. Naturally you can have multiple OS processes of Python that run in parallel. This is how Python web services usually work. Then, of course, you need to share memory explicitly via some mechanism like Redis or SysV shared memory, so it leads to different design considerations.
  - ynik 5y ago
    You can only have a single core running Python code per process. You can also have a single Python process running multiple threads in parallel, as long as all but one of those threads are currently running C code (numpy, system calls, ...) But if your program spends more than a tiny fraction of time in interpreted Python code, the GIL will slow you done if you try to use all cores. You can have multiple processes running Python at the same time -- and indeed, many Python programs work around the GIL by forking sub-processes. This tends to introduce significant extra complexity into the program (and extra memory usage, as the easiest solution is typically to copy data to all processes).
    - Symmetry 5y ago
      If your inputs are immutable and you don't want to copy them that works too since if you're careful. Copy on write means that it won't be duplicated by default. Of course reference counting changes means you will be writing to the inputs by default but if you're just branching for a computation you can temporarily turn off garbage collection while you diverge. Disabling garbage collection for a short lived forked process can even make sense on its own terms.
  - masklinn 5y ago
    > TFA makes it sound like no two Python programs can execute simultaneously on two different cores It does not. It states that two python threads can’t execute bytecode simultanously, the implication is that that’s within a single process (which is the case) otherwise talking about threads makes no sense. > I have the feeling what the TFA really means is something different. But I cannot really figure out what. TFA means exactly what it says. Two threads can’t execute python bytecode in parallel, because there’s a big lock around the main interpreter loop (that is in essence what the GIL is). That is quite obviously within the confines of a single Python interpreter = runtime = process.
- funklute 5y ago
  > Using it for actual production code makes about as much sense as writing production stuff in bash This is a really strange perspective, given how many shops use python for production these days. > Ruby is a bit better, but these days there's no reason not to use Go or Rust. Unless you're specifically looking to access for example machine-learning resources, that are not nearly as mature in those other languages.
- lmm 5y ago
  > if you look at the history of Python it was supposed to be a teaching language No it wasn't. It was supposed to be a language for automating sysadmin scripts, or for embedding in a program to run user-scripts.
- tored 5y ago
  PHP can be built as either thread safe (TS (the term ZTS, Zend Thread Safe, is also used)) or non thread safe (NTS), depending how you deploy PHP (how the webserver interacts with PHP like FastCGI or SAPI, if it spawns processes or threads) you run one or the other, most common is that you use NTS builds on Linux and TS builds on Windows. PHP does not have a GIL, thus PHP does not suffer from any slowdown when using threads, proper use of threading improves PHP performance without the need to escape to a C library. Internally PHP uses something called Thread-Safe Resource Manager (TSRM) when built as thread safe. TSRM architecture was improved with PHP 7 and to my understanding performance as well. However PHP does not ship with any API to handle threads, this is because normally you, the web programmer, don't do any process/thread handling in a web setup, concurrency is handled by the webserver for each request. It does exist third party extensions that enables threading API, like in a cron job.
- joconde 5y ago
  > Using it for actual production code makes about as much sense as writing production stuff in bash, you can do it but you're just making things harder on yourself for no good reason. Even Ruby is a bit better, but these days there's no reason not to use Go or Rust Good luck doing any deep learning in Go or Rust :)
  - pjmlp 5y ago
    No problem with the "Python" libraries that are actually C, C++ or Fortran code.
CrLf 5y ago
If you are CPU-bound in pure Python code you’ve already lost. However, most of the time you’re doing CPU-heavy tasks in Python you’re likely using libraries that are ultimately implemented in another language for performance. The GIL can be (and is) released in those cases. Not much of a problem there. It’s also released when you’re waiting for I/O. Not much of a problem there either. Fear of the GIL have never stopped me from using threading in Python, and it has never been much of a problem. In the rare occasions I’ve noticed, I was doing something wrong, not using the appropriate library, or both. Or it might be the case that Python was just a poor choice for the problem, no matter how flexible it is. I also wonder what would happen if CPython got rid of the GIL. Python has certain characteristics that make it hard to have a parallel interpreter, and people have tried (and failed) to remove te GIL before. Don’t fear the GIL.
BiteCode_dev 5y ago
This is for real, but this doesn't mean you can't use parallelism. Only that you can't use thread for them most of the time . Parallelism strategies in Python usually revolve around using the multprocessing module, a c extension (numpy will release the gil, and hence all the science stack) or using a 3rd party tools (dask is pretty popular, and allow easy parallelism: https://dask.org/ https://dask.org/).
- bobbylarrybobby 5y ago
  Even python's parallelism is subpar because objects passed across processes have to be pickled. This both takes time and limits the kinds of objects that can be sent across processes (no lambdas, for instance).
fastball 5y ago
No, because in a webserver environment where you have 4 cores, you should be running 4 instances of your program, load-balanced behind nginx or caddy or w/e. Also what's your source for the claim that Python is 6x slower than JS or PHP?
quietbritishjim 5y ago
> The GIL allows only one OS thread to execute Python bytecode at any given time [within each OS process] Different Python processes can certainly use different CPU cores - the sentence is definitely misleading. (As others have mentioned, it also only applies to interpreting Python byte code itself, not OS calls or C extensions modules called from that bytecode.)
jrochkind1 5y ago
Ruby has the same issue. But a "web server environment" scales pretty well "horizontally", meaning in ruby the solution/workaround is generally to run 4 separate web worker processes if you have 4 cores, so they won't be limited by the shared GIL. I am a rubyist and don't know much about python, but I'd assume python web server environments do the same? Is this as good as a high-performance concurrency platform (like, say, the JVM has become)? Will it lead to absolutely optimal utilization of all cores? No. But the result is not simply "24 times slower", and all cores will be in use. Performance numberes like "6x slower" are almost always contextual when it comes to the real world, it's not like every single program written in python takes 6x longer/6x as much CPU as the nearest-equivalent program in PHP, that's not how it works, it will depend on what the program is doing and how it's deployed.
acdha 5y ago
The problem is that performance is variable. For example, it's not just wrong but meaningless to say "6x slower” because there are cases where Python is either faster or slower than PHP or JavaScript and you need to know what specific things you're comparing it to provide a number. For example, I've seen people write entire programs in C and, after taking several weeks, realize their code was nowhere near as fast as the Python version because the code was bottlenecked on dictionary operations and Python's stdlib version was far more tuned than the C implementation they'd written (repeat for image handling where they wrote their own decoder, etc.). This is not to say that a C programmer shouldn't be able to beat Python — only that performance is not a boolean trait and it's important to know where the actual work is happening. The important thing to know is that the GIL can be and is commonly released by C extensions. If your programs tend to be I/O bound (e.g. many web applications) or are using the many extensions which release the GIL as soon as they start working, your performance characteristics can be completely different from someone else's program which is CPU-bound and has different thread interactions. This is why you can find some people who think the GIL is a huge problem and others who've only rarely encountered it. These days, I would typically move really CPU-bound code into Rust but I would also note that in the web-ish space I've worked in the accuracy rate of someone blaming the GIL for a performance issue has been really low[1] compared to the number of times it was something specific to their algorithm. 1. I'm reminded of the time someone went on a rant about how Python was too slow to use for a multithreaded program working with with gzipped-JSON files — 3 months and an order of magnitude more lines of Go later they had less than a 10% win because Python's zlib extension drops the GIL when the C code starts running and was about the same speed as Go's gzip implementation, both running much faster than storage. The actual problem was that the Python code was originally opening and closing file handles on a network filesystem for every operation instead of caching them.

fastball 5y ago

For anyone looking for something on the same topic but a little bit lighter / more practical, I really enjoyed this talk[1] by Raymond Hettinger at PyBay 2017, where he discussed the GIL as well. [1] https://www.youtube.com/watch?v=9zinZmE3Ogk https://www.youtube.com/watch?v=9zinZmE3Ogk

TheChaplain 5y ago

As an avid user of Python I can calm any readers that the GIL issue will not have any effect on a considerable majority of your use cases. Python is not broken but works very well and helps people make millions all over the world.

matsemann 5y ago
I've been bitten by the GIL multiple times since switching to a company writing mostly python. Not that I write that much multithreaded code, but the GIL influences everything so lots of patterns in other languages no longer work. For instance, a local cache of some values is basically useless when python is used as backend. Since there will be multiple processes handling the requests (in the same node), they don't share memory. So basically everything has to be stashed at redis. Of course, with scale and multiple nodes lots of stuff should be cached in redis anyway. But not everything makes sense to reach out to an external service for. Like fetching a token I need to use multiple times, instead if having it stored locally in a variable. There are of course ways around this, using some locally shared file or mmap or whatever, but it's a hassle compared to just setting a variable. Same with background tasks or spinning up extra threads to handle the workload of an incoming request. Can't easily be done, so basically every python project includes celery or something similar. Which works fine, but again, a different and forced way to solve something. Asyncio solves many of the daily problems I would have used threads for, though. As I often used it other places to fire off some http requests to other services in parallel. But unfortunately async doesn't work properly with everything yet. So everything has a solution, but you are forced to do it "the python way" and with some hurdles.
- pjmlp 5y ago
  What about doing it the UNIX way before pthreads became a thing?
  - gpderetta 5y ago
    Well, yes, but there is a reason why pthreads became a thing.

deleted 5y ago

[deleted]

happy_path 5y ago

Weren't a Python project called interpreters that had the aim of allowing different interpreters of Python, thus mitigating the the issues the GIL causes?

BiteCode_dev 5y ago
Yes, a first version has been released, but it still needs a lot of ironing.
- happy_path 5y ago
  Awesome! Would it be ready for the next version?
  - BiteCode_dev 5y ago
    Unlikely. The module is usable, and you can already import it and run it. But the perf benefit are still not there.
    - germandiago 5y ago
      What is this module about actually? I mean, is it a model to run code in parallel in some way different to multiprocessing?
      - BiteCode_dev 5y ago
        For now it's only at the c level, with a python api in discussion: https://www.python.org/dev/peps/pep-0554/ https://www.python.org/dev/peps/pep-0554/ The idea is that the python code interpreter would not rely on global variables anymore, so that you can instantiate it several times. The GIL is tied to the interpreter, meaning 2 threads in 2 different interpreters do not share a gil, and can be schedule on different CPU.
frakkingcylons 5y ago
That project is mentioned in the article.

est 5y ago

tl;dr don't use threading in CPython unless you know what you are doing. In fact projects like uWSGI disables _thread.so by default.

BiteCode_dev 5y ago
Rather, don't use threading for the sake of parallelism in Python. Threads in Python are mostly useful to avoid blocking: - you need to make io calls and don't want to bring asyncio - you need to process things in the background without blocking the main thread (e.g: zippping things while you use a GUI) - you want to run a long process on a c-extension based data structure (they release the gil, so you get the perf boots), such as a numpy array or a pandas DF

___luigi 5y ago

Is there any reference to delve in the details of GIL, multithreading, multiprocessing, high performance computing on top of Python eco-system?. There are some interesting projects such nimba, Dask, etc, but the resources are a bit scattered.

xscott 5y ago

I don't understand why people care about the CPython GIL. For computationally intense stuff (numerics) running in the interpreter, the language is generally 60 times slower than PyPy and often 100 times slower than an equivalent C program. That means if you want performance, you would need to dodge Amdahl's law and have 60-100 processors for a hypothetical GIL-less CPython to match a single threaded program in PyPy or C.

zmmmmm 5y ago
If all it was used for was scripting it would be fine, but what is strange is that Python has emerged as a major language for complex data analysis. So I am sometimes confronted with partial solutions in Python that solve a piece of the puzzle that is purely numerical (so all runs on numpy etc) but then extending it to actually solve the real problem ends up either extremely awkward (the whole architecture getting bent around Python's limitations) or being a complete rewrite. It's frustrating that the ecosystem for certain problems is so bent towards a language that is unsuitable for large parts of the job. (mainly, I would add, because it sucks the oxygen away from alternatives that would be better in the longer term).
- xscott 5y ago
  > it sucks the oxygen away from alternatives that would be better in the longer term It's always frustrated me that the computing world isn't as it should be. Python as a language seems to have found a sweet spot for accessibility to a lot of people, but CPython the interpreter steals all the air from PyPy, which is a vastly better implementation.
  - seunosewa 5y ago
    PyPy has a GIL too though.
    - xscott 5y ago
      The PyPy FAQ addresses this. To my eyes, it looks like this is partly to provide the same promises CPython does for C extensions, and partly because their transactional memory approach hasn't gotten the attention it would need to succeed. I think both of those show CPython stealing the air from PyPy. https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...

r4victor 5y ago

Hi! I'm the author of this article. Thanks for posting it. The GIL is an old topic, but I was surprised to learn recently that it's much more nuanced than I thought. So, here is my little research. This article is a part of my series that dives deep into various aspects of the Python language and the CPython interpreter. The topics include: the VM; the compiler; the implementation of built-in types; the import system; async/await. Check out the series if you liked this post: https://tenthousandmeters.com/tag/python-behind-the-scenes https://tenthousandmeters.com/tag/python-behind-the-scenes I welcome your feedback and questions, thanks!

sirfz 5y ago
Haven't checked the full series yet but this GIL article is very nicely written and insightful. Thank you
jeroenvlek 5y ago
You filled in a lot of gaps in my GIL knowledge. Today I learned something new. Thank you for writing it!
gadrev 5y ago
Wow, just skimmed over a few articles in the series and they seem very nice and well researched. Congratulations.
harpiaharpyja 5y ago
In the opening paragraph you state that the GIL prevents speeding up CPU-intensive code by distributing the work among multiple threads. My understanding is that distributing work across multiple threads would not speed up CPU-intensive code anyways. In fact it would add overhead due to threading. Perhaps you meant I/O bound code?
- jjice 5y ago
  If it was being distributed amongst python threads (which run on one hardware thread), then CPU performance can't improve since they're just taking turns using the CPU. If you're running on multiple hardware threads (what I assume the author meant), that can causes better CPU performance since it will distribute work across real threads that can run in parallel. The GIL restricts use of multiple hardware threads.

helpfulmandrill 5y ago

Ruby seems to have got past its GIL problems with Ractor[1]. What's stopping Python from doing something similar? [1] https://scoutapm.com/blog/ruby-ractor https://scoutapm.com/blog/ruby-ractor

r4victor 5y ago
As far as I understand, Ruby's Ractors are what Python is now trying to do with subinterpreters (https://lwn.net/Articles/820424/ https://lwn.net/Articles/820424/). I mention them in the post.
- danachow 5y ago
  I just skimmed through your link. Subinterpreters have been in the C api for a long time and never got around the GIL, and reading over the post that you linked — nothing has really changed.
  - r4victor 5y ago
    This is a WIP. Here's how it's going on: https://pythondev.readthedocs.io/subinterpreters.html https://pythondev.readthedocs.io/subinterpreters.html
georgyo 5y ago
Python has multiprocessing which also bypasses the GIL. It also works with message passing.
- helpfulmandrill 5y ago
  Much more heavyweight than Ractors though, right?
- mzs 5y ago
  Which was broken on macOS until 3.8 and is fragile in the face of threading and freezing.
5y ago

jrochkind1 5y ago

> Literally, there is an initiative to introduce multiple GILs to CPython. It's called subinterpreters. The idea is to have multiple interpreters within the same process. Threads within one interpreter still share the GIL, but multiple interpreters can run parallel. No GIL is needed to synchronize interpreters because they have no common global state and do not share Python objects. This sounds very much like Ruby's recently introduced "ractor", no?

aynyc 5y ago

For those who find GIL impacts their developments, can you post some examples? I'm curious as to what they are.

jillesvangurp 5y ago
Well many developers end up using different languages for this reason. E.g. Go and node.js are popular and both do asynchronous stuff effortlessly (even though they are single threaded typically). And if you use Jython (python on the jvm), you can actually do threads properly as the Gil is not a thing there: https://stackoverflow.com/questions/1120354/does-jython-have-the-gil https://stackoverflow.com/questions/1120354/does-jython-have... Likewise, Iron Python also has no Gil apparently. Both are running quite a bit behind PythonC at this point unfortunately. For properly multi threaded stuff, people use more higher level languages like Erlang, Scala, Java, Kotlin, Rust, C++, etc. In general batch processing data is both IO and CPU intensive (e.g. parsing, regular expressions) and a common use-case for Python. You could say, data science is actually the primary use case driving its popularity. Python is effectively single threaded for that stuff. Using multiple processes fixes the problem well enough though. Conveniently the module for using processes looks very similar to the one for using threads, so switching is easy once you figure out why shit doesn't scale at all (been there done that). But using processes also adds some complexity. The good thing is that it makes doing why the Gil is needed (sharing data between threads/processes) a lot harder/impossible. The bad news is that it's hard. Typically people use queues and databases for this. ETL systems like Airflow tend to use multiple processes and queues instead of threads as using python threads is kind of pointlessly futile (from a making things faster point of view). However, with Airflow a pattern that you see a lot is that python process workers are typically not used and instead it is configured to use one of several cloud based schedulers (e.g. kubernetes, ECS, etc) to schedule work. Basically, airflow is just used to orchestrates work where the actual work happens elsewhere and may or may not involve python code. A lot of the performance critical stuff is native anyway.
- doctordesh 5y ago
  > ... Go and node.js are popular and both do asynchronous stuff effortlessly (even though they are single threaded typically) I know that I'm being very pedantic here, but Go is not a "typically single threaded" language. Go has as fine control over it's threads (goroutines) as C does over it's threads. Go is very capable of true parallellism.

Mikeb85 5y ago

Honestly, not sure why people care so much about the GIL in languages like Ruby, Python or JS. Let's face it, nowadays when you deploy software it's over multiple machines, nodes, whatever you want to call it. So you're almost always talking about spawning processes. Which any language can do, GIL or not. Removing the GIL to enable parallel processing on a single machine adds more complications than it's worth, especially when any Python workload is either going to be forking C processes or forking Python processes. Not every language needs the same kind of control over OS threads of a C, C++ or Rust.

zamadatix 5y ago
People don't care as much about removing the GIL in JS because JS never exposed threads. When the time came for spreading compute load to multiple cores JS decided to simply create multiple contexts first with just message passing and then later with shared memory. As a result you can still code JS in the typical thread unaware way but utilize multiple cores without having to go multi-process or multi-machine. For in thread stuff the async model (or historically callbacks) is good enough if you're not compute bound. In Python threads are exposed but only one runs at a time and you have to go full multiprocessing to use the other cores. This is both confusing and annoying respectively hence the increased noise about the GIL.
cma 5y ago
We already have 64 core/128 thread prosumer machines and it is only going to be more in the future. Python is so slow that message passing between processes can often be slower than just handling it in the same process. What if you have a big array of python objects that you want to sort? In other languages it is trivial to get a huge speedup with more cores.

cmacleod4 5y ago

The multiple interpreters communicating by message passing model mentioned towards the end of this article as a way forward is exactly the threading model which Tcl has been using successfully for the last 20 years. For more info see https://www.activestate.com/blog/threads-done-right-tcl/ https://www.activestate.com/blog/threads-done-right-tcl/ and https://www.tcl.tk/man/tcl/ThreadCmd/thread.html#M44 https://www.tcl.tk/man/tcl/ThreadCmd/thread.html#M44 .

celeritascelery 5y ago
Are tcl functions shared between threads? Or does each new thread interpreter start from a clean state?
- cmacleod4 5y ago
  Functions (procs) and other state are not shared between interpreters. So they do need to be reinitialised in each thread. This is not hard to do though. The messages which can be passed between threads can be arbitrary scripts to be executed in the target interpreter, such as proc definitions.
- blacksqr 5y ago
  The Tcl thread library allows you to create multiple threads in a single interpreter, and use mutexes. locks etc. to manage them. In this case of course you only need to initialize your interpreter once. The library is most used in an "easy" mode however, where each thread starts up with its own interpreter and thus the usual mutex/lock dance is unnecessary. Various forms of message passing can be done among these interpreters, including tools provided by the thread library itself, Tcl's built-in synthetic channel feature which makes communication look like standard file I/O, or, since each interpreter has its own event loop, Tcl's socket I/O features can be used to set up server/client comms.