11 ms·

GIL removal and the Faster CPython project

339 points by signa11 3y ago

nullspace 3y ago
What a great article from LWN. It was well-worth reading. As someone who was excited about the NoGIL from Sam Gross when it was first posted here, I think I'm beginning to change my mind after reading this article and reflecting on my own personal experiences. My experience is with writing backend systems in several different languages (including Python) at various volume/latency/throughput levels. I've basically worked on only two types of systems - 1. one that exposes some sort of an endpoint to the network - it accepts requests of some kind, does computation and other network requests and sends response of some kind (including long polling, ws etc). 2. reads a message from a "queue" (could be database, could be based on polling another api etc) and does computation/network calls and basically sends it to other queues. Nothing else. Huge variance in specific requirements, but that's it. For the first type of system, latency matters more. For the second system, throughput matters more. For the first type of system, I want to be able to spin up threads in response to requests, without worrying that an endpoint is too computationally heavy and might block others. I want to be able to share connections to databases in a shared pool. NoGIL would be useful here. For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines. I would absolutely be disappointed if NoGIL meant compromising on the quality of of the second type of system here. In practice, most of my mental bandwidth today goes towards making the second type of system better.
- YeBanKo 3y ago
  To take advantage of NoGIL you don’t necessarily need to use parallelism directly. But let’s say your web server or async task executor can be more efficient at sharing context between threads.
- VagabundoP 3y ago
  As a hobbyist who uses python I don't think I'll be directly using concurrency in my code, but I'm betting that over time the standard library and popular external libraries will. And that will raise everyone's code.
- scarygliders 3y ago
  > For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines. For myself, the prospect of no-gil is interesting, in that something like my Captain's Log application [0] can be free from it; for example, I currently use a QThread to implement a JournalParser, which is basically the program's "engine" - the parser constantly reads in game events from a player journal file generated by the game Elite: Dangerous (and Odyssey), and depending on the particular event, fires off a related custom QSignal, which is then processed by whichever slot (receiving function) is listening for a given Signal. There are other places in that application where no GIL might be quite handy. In other words, I can see where having no GIL can be useful for GUI applications like mine. [0] https://captainslog.scarygliders.net/captains-log-2/ https://captainslog.scarygliders.net/captains-log-2/
  - radarsat1 3y ago
    Your JournalParser sounds like it could be implemented using normal Python threads or by an asyncio event loop without much performance problems. If I understand all it's doing is watching for events and posting a signal somewhere, so it doesn't sound like the kind of application that is CPU-bound.
    - scarygliders 3y ago
      I could. But I 100% take full advantage of Qt's signal and slot mechanism. Also, in many ways, GUI applications written in Python are not so much CPU bound, but Python GIL bound. If you're writing a Python/Qt application, you have to take great care to ensure your GUI doesn't freeze when your program is performing, say, many database inserts; if you have some naive loop which performs some given operation, your nice Qt GUI will freeze right up until that operation is complete. Right now the solution is to perform such operations in, say, a QThread, and use Qt's signal/slot feature to blat a progress "report" to a handler in the `main` Python loop. So back to what I said - no-GIL is looking quite interesting to me. Whether or not Qt can take advantage of such will be a different matter.
- spacechild1 3y ago
  The GIL is a bottleneck in applications that are CPU bound, e.g. machine learning, so naturally the NoGIL project is not that interesting to people writing server applications. Of course, one may argue that you probably should not write CPU bound programs in Python in the first place, but that's another story :)
  - jillesvangurp 3y ago
    A lot of Java server based applications are multi threaded, not CPU bound, and tend to do a lot of things that generally can't work in python because of the GIL. It's too simplistic to think of this as something only of interest for CPU bound stuff. A lot of what Java applications do is of course the thread per connection style processing that older java applications still do (more modern ones would use non blocking IO and green threads). But there are also background threads doing useful work or more complex requests that fork off asynchronous work across multiple CPUs and then aggregate the results back as the response. Java apps tend to have vastly more threads than CPU cores. The exception is when things are CPU bound; then you want to minimize the context switching and end up with an number that is close to the number of CPU cores. The GIL is not about the CPU but about enabling those kinds of things. With the current GIL in place it's very simple: as soon as you hit the global lock, everything stops until it is released. It doesn't matter how many CPU cores you have, they'll be idling while one of them holds the lock. There's barely any point in even trying to do that with the GIL in place. Forget about sharing data between threads. Mostly that's done via queues or databases in python. Removing the GIL will revolutionize a few things in key use cases for python: - data processing & ETL - event driven server systems - machine learning and data science systems They can all benefit from this and that's the reason a lot of people are pushing for this. The short term performance losses are not inherent to removing the GIL but just a necessary evil while the python developers deal with fixing the bottlenecks and a few decades worth of technical debt.
    - spacechild1 3y ago
      I/O functions (may) internally release the GIL. If the GIL becomes a bottleneck, you are not I/O bound by definition. However, you are certainly right that not all server applications are I/O bound. I was a bit sloppy there.
- mirsadm 3y ago
  >For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines. Interestingly I'm working on something like this right now and do have large shared resources which meant I had to abandon using a multiprocess strategy. I don't see why it would be confusing though, provided the shared resources are read only.
  - nullspace 3y ago
    For such applications, isn’t the parallelism count usually static and limited? I think there will be good benefits for distributed system frameworks for python. Id agree. A couple of years ago, I implemented in process parallelism for a system I was maintaining at $JOB. I was happy the system was in Go and not Python. But it was an exception to the rule in my experience.
w10-1 3y ago
I can't imagine a more difficult test case for open source. The decision is entirely reasonable, to move forward in test mode, treat multiple interpreters as an interim experiment, but target being concurrent. The constraint of running old and new code in the same VM is a tall one. Surprisingly, LWN's summary says nothing about testing, which is largely unsolved and could lead to releases with unknown but serious bugs. Microsoft, Facebook/Meta, and Conda have stepped up with resources and a super-majority of core contributors want to move forward, but it's unclear what happens if things get hard and more people are needed. Meanwhile a crazy number of projects in academia and industry from web sites to big data to AI depend on Python. The potential to export costs to python developers might be measured in percentage of GDP. It doesn't sound like the problems are even known yet. It might be fairer to commit to the Faster CPython approach of knowable improvement over the next 3+ years, but have the concurrent Python promotors do more than just prototype. They should analyze the kinds of problems that could present, how they could be detected, and what can be done about it. This should be reviewed by people with backgrounds in proving concurrency guarantees. Then the questions can be fairly presented to the steering committee, when the unknowns are at least identified. There's not a lot of program-management scale decision-making in open-source, but most have been relatively simple questions of driving the market in a given direction (apache, eclipse, linux). This has real technical unknowns. And I can't help feeling they should also address inter-language ABI's at the same time. A big issue is matching the expected execution model of C. Java's foreign-function and memory interfaces have been incubating for many years, and Swift is also getting better at wrapping C and C++, but FFI's are notoriously (and likely unnecessarily) difficult.
- michaelt 3y ago
  > The potential to export costs to python developers might be measured in percentage of GDP. Of course, that's also true of the benefits.
BiteCode_dev 3y ago
The article is well written, and a good history of the all ordeal. But please note it gives more weight to the "against the GIL" side of the story, and all that can go wrong. It doesn't highlight enough the other side of the coin: - Sam's work is very high quality, and he brought with the no-gil some unrelated perf improvements so that people don't feel like they loose perf too much. - Sam played the open source game perfectly, and is incredibly patient given what he is bringing on the able and how slow and flaccid the steering council reaction was (without the community pushing on it, it would still be collecting dust). - Sub interpreters have yet to demonstrate any usefulness at all in Python. In fact, any serious metrics at all. This is the first attempt to be that well defined, and measured. - The community feedback shown a great interest in this particular project. - The steering council did conclude "We intend to accept PEP 703, although we’re still working on the acceptance details." I'm not a no-gil enthusiast. I would be fine with have it never be removed, and I think we should try sub-interpreter first. But what's fair is fair.
- TylerE 3y ago
  What I’d liked to see is the performance patches without the no-Gil part.
  - Znafon 3y ago
    The performance patches have already been merged and there is ongoing working in the Faster CPython project.
  - formerly_proven 3y ago
    You're already using a bunch of those in recent Python versions.
- Znafon 3y ago
  > I would be fine with have it never be removed, and I think we should try sub-interpreter first. There is a lot of work going on with sub-interpreters and the per-interpreter GIL is shipped in Python 3.12. The results are very impressive: https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/ https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/, as good as one could have hoped I think. It seems to me like the work on sub-interpreters will continue in parallel ;) to the work on free-threading. Sub-interpreters and no-GIL have different use-cases though.
zarzavat 3y ago
It’s more important to prioritize single-threaded performance because it’s much harder to improve by throwing money at the problem. With multithreaded performance, you can just add another core to (more than) offset whatever overheads there are from using process-based parallelism. I think that this entire GIL vs No-GIL dichotomy is misguided. The biggest problem people have with multiprocessing is that you can’t share memory. So add virtual processes with an explicit mechanism for memory sharing. Then you can keep all of your single-threaded optimizations like refcounting without barriers because the objects for one thread will stay in that thread.
- Galanwe 3y ago
  Agree 100%. If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. Fixing _that_ should be the objective. What Python need is better support for placing native instances in shared memory.
  - Znafon 3y ago
    > If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. > The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. The PEP goes into the motivation behind this work, and using multiple process does not magically solves all the issues: > Multiprocessing, with communication via shared memory or UNIX sockets, adds much complexity and in effect rules out interacting with CUDA from different workers, severely restricting the design space. > I reimplemented parts of HMMER, a standard method for multiple-sequence alignment. I chose this method because it stresses both single-thread performance (scoring) and multi-threaded performance (searching a database of sequences). The GIL became the bottleneck when using only eight threads. This is a method where the current popular implementations rely on 64 or even 128 threads per process. I tried moving to subprocesses but was blocked by the prohibitive IPC costs. > NumPy does release the GIL in its inner loops (which do the heavy lifting), but that is not nearly enough. NumPy doesn’t offer a solution to utilize all CPU cores of a single machine well, and instead leaves that to Dask and other multiprocessing solutions. Those aren’t very efficient and are also more clumsy to use. That clumsiness comes mainly in the extra abstractions and layers the users need to concern themselves with when using, e.g., dask.array which wraps numpy.ndarray. It also shows up in oversubscription issues that the user must explicitly be aware of and manage via either environment variables or a third package, threadpoolctl. The main reason is that NumPy calls into BLAS for linear algebra - and those calls it has no control over, they do use all cores by default via either pthreads or OpenMP. and it discusses the alternatives at https://peps.python.org/pep-0703/#alternatives https://peps.python.org/pep-0703/#alternatives.
Mayzie 3y ago
I understand the desire for multi-threaded CPython scripts, but why not do a PHP5 -> PHP7 and focus on single-thread performance more, particularly around implementing a JIT? We know from PyPy (and other JIT-enabled languages like NodeJS) that performance improvements can be significant. I feel like that should be the primary focus for speedups, and have always wondered why the idea has never been more popular. Has a CPython JIT been ruled out for some reason I'm not familiar with?
- turndown 3y ago
  You should look into the copy & patch efforts underway for Python[0]; an actual JIT will probably never exist but I think c&p has a shot of being mainlined in the next few years, such that Python could dynamically choose to either run the interpreter or a c&p option. 0: https://github.com/faster-cpython/ideas/issues/588 https://github.com/faster-cpython/ideas/issues/588
- dagw 3y ago
  Has a CPython JIT been ruled out for some reason There have been several attempts at integrating a JIT into CPython. Google made a stab at it back around version 2.6 and there was a second attempt around the 2->3 change to build a JIT on top of LLVM. However none of these projects produced results that the core developers felt were good enough across the board and had problems with backwards compatibility and lot cross platform LLVM issues (as it was a pretty new project) so they were dropped after Google stopped funding the project. That being said there is still work being done and people are hoping for at least some JITing in upcoming python. Pyston is project being worked on Guido Van Possum himself and both Microsoft Instagram have their own JITed versions of Python they use internally. Some initial parts of Pyston are even scheduled for being included in python 3.12 or 3.13
- josefx 3y ago
  PyPy breaks compatibility with CPython in significant ways, as far as I understand the current no-gil proposal only got through because it has an automated fallback for modules that are not no-gil compatible.
  - 3y ago
VagabundoP 3y ago
Excellent write up as usual from LWN. I love the Python community. It's really a leading light for open source software. And it shows what transparency and good governance can achieve. Although I appreciate the engineering hours that Meta, Microsoft and others give, its pretty miserable still compared to the value that the whole tech industry (and beyond with data science) extract from Python and other open source software.
- stevesimmons 3y ago
  We can all help to change that, within our own organisations! I did my bit at JPMorgan 8 years ago, convincing the tech leadership team to sponsor PyCon UK, plus a recruiting stand and supporting a group of junior developers from all JPMorgan's UK locations to attend. I left JPM 5 years ago now, and they are still PyCon UK's headline sponsor. Compared with the enormous benefit we got from Python and its open source ecosystem, it was a totally negligible cost.
- lrtzyv 3y ago
  They pretend to be transparent. All actual decisions are made in backrooms. The mailing lists are censored and the inner circle cannot be criticized. Real contributors are exploited by those who work for the right corporations, do very little and go for any clerical position of power. Do not be misled by the LWN articles, which are very kind and always namedrop the deciders. It is selective reporting.
  - nojonestownpls 3y ago
    > The mailing lists are censored and the inner circle cannot be criticized. Well, HN isn't censored in this regard (unless you believe their influence extends here too). What specific criticisms do you have against "the inner circle"? And what evidence can you show for them?
TekMol 3y ago
There was a question from Shannon about ""what people think is a acceptable slowdown for single-threaded code"" ... he had estimated an impact "in the 15-20% range" Horror! To me, the acceptable slowdown is exactly zero. I can already use multiple cores by running multiple Python processes. Any slowdown of single-process performance would be a terrible step backwards. Python is already slow. It should look at PHP and see what it can take out of their book. PHP 7, which had no JIT, was already about 6x faster than Python. PHP 8 is even faster for some workloads. I'm not sure if the overhead of a JIT makes sense though. But PHP 7 is a good place to look for a performance benchmark.
- deleted 3y ago
  [deleted]
- dagw 3y ago
  To me, the acceptable slowdown is exactly zero. This has been the reason most of the previous GIL removal projects failed. GvR had a strict zero slowdown in single-thread performance rule for any nogil patches and none of them managed that. I wonder if the core team has decided to abandon that rule now that GvR is gone?
  - alecmg 3y ago
    I saw comments that if introduced together with Faster CPython single thread performance optimisations, nogil would end up no slower than previous version. So it is best time to implement
- ohgodplsno 3y ago
  >I can already use multiple cores by running multiple Python processes. Because you have terribly simple processes that don't need to do synchronization. Even a simple parallel map would have Python scream and kick, because the GIL prevents it from having any kind of reasonable performance, and your multiprocess can't handle that unless they all write to files.
- pkolaczk 3y ago
  > Horror! To me, the acceptable slowdown is exactly zero. If you care about single % of performance, why are you using Python in the first place? You're likely losing 100x to more performant languages already.
globular-toast 3y ago
Multi-threading gets you programs that are, at the very most, 8x faster. Is it really worth it? If this makes it even slightly more difficult to develop Python for normal people then it will get forked, and that sucks. If it can be done without any impact then great, but I'm still not sure it's worth it.
- josefx 3y ago
  > at the very most, 8x faster. Where does that number come from? My current desktop has 32 logical cores and it has been a few years since I bought that one.
  - alecmg 3y ago
    maybe its a reference to Amdahl's law? If you can parallelise 90% of your code, you get only 10x improvement even on infinite number of cores
    - josefx 3y ago
      You mix up time and code. You don't have to parallelize 100% of your code to scale with cores, you only have to parallelize the code that your software spends most of its time executing and that is often significantly less than 90%.
- usrbinbash 3y ago
  > at the very most, 8x faster. My 64 core server setup says otherwise. > Is it really worth it? Yes, very much so. > If this makes it even slightly more difficult to develop Python for normal people It won't. Pure python code will very likely not notice a difference.
coldtea 3y ago
>There was a question from Shannon about "what people think is a acceptable slowdown for single-threaded code". To a large extent, that question went unanswered in the thread, but he had estimated an impact "in the 15-20% range, but it could be more, depending on the impact on PEP 659". So, let me get this straight, some of those working in the project to "make CPython faster", think it's acceptable to overnight make most existing Python code 15-20% slower? I'd say max 5% and that if the gil removal was a benefit to other optimizations going forward (they say the opposite: the change with complicate and stall their other optimizations). Meanwhile, Shannon had a fundraising proposal for him personally "speeding up CPython" 5x back in 2020. Now it's a whole team working on speeding up CPython with much larger corporate support, and it seems their targets are quite smaller?
- tda 3y ago
  But consider that in perhaps 99% of current python code high performance (of the pure python code) is not the primary concern. And now for the times when performance does matter you can get a 20x (?) speedup without rewriting in another language.
  - coldtea 3y ago
    I'd expect in many those cases people care for performance, they already use multiprocess (so the impact from a 20% hit GIL-free CPython would actually be a slowdown). And for those that do not use multiprocess, rewriting their currently single threaded Python code for no-gil shared-concurrency will indeed bring a speedup, but would still be a significant rewrite. Also there are people using Python for things like web serving, where performance might have been secondary to convenience when they picked Python, but still a sudden 15-20% slowdown will seriously impact their server budgets (or ability to update versions)...
  - School-Cotton 3y ago
    > But consider that in perhaps 99% of current python code high performance (of the pure python code) is not the primary concern. Then why do we care about GIL removal at all?
- 3y ago
londons_explore 3y ago
I want to see python with automatic and transparent concurrency. Ie. The programmer writes: for file in glob.glob('*.jpg'): data.append(load_file(file)) Then python itself parallelizes that work. It should do that by identifying loops of pure functions and pushing them off to other threads. One should also be able to mark a function as pure. Python itself would be responsible for making everything appear as if the code ran serially - for example by buffering/reordering log statements. Rationale: Python is designed to be easy and simple. Concurrency in general is not that. Python should do only do concurrency in a way that can be made easy and simple.
- lmeyerov 3y ago
  Not sure if intentional: That's a super frustrating program for a traditional automatic parallelization compiler, e.g., polyhedral: * glob is an i/o operation: impure * load_file is an i/o operation: impure * data.append serializes part of the execution order --- Interestingly, GPT4 does pretty ok: files = glob.glob('*.jpg') with ThreadPoolExecutor() as executor: data = list(executor.map(load_file, files)) --- Separately, if just going for concurrency for i/o, async/await is pretty amazing: tasks = [load_file(file) for file in glob.glob('*.jpg')] data = await asyncio.gather(*tasks) A lot less program transformation needed as it's essentially sugar over promises, and avoids much of the need to restructure code beyond the coloring problem. So I'm not as much about the case for a compiler here..
  - londons_explore 3y ago
    > not sure if intentional. It sure was! I believe any automatic parallelization should be able to deal with all these things. With a combination of opportunistic execution (ie. guess which bits can be parallelized, and roll back if wrong), and clever heuristics to avoid rollbacks, it should be pretty do-able. CPU manufacturers do similar stuff at a smaller scale with parallel execution of single threaded code with Tomasulo's algorithm, and they get massive gains compared to non-superscalar CPU's. They even do similar stuff at medium scale with Hardware Lock Eliding.
  - starlevel003 3y ago
mihaic 3y ago
Great exposition of the situation! I was in the no-GIL camp until a few years ago, until I started thinking of all the implications: I have tons of code that modifies global maps from all over the place, and I feel relieved to not manage threads like in C/C++. What I hope the solution will end up emerging would be subinterpreters that are more intuitive to use. Honestly, I don't want to pass in source code, but rather a function or module entrypoint and have the subinterpreter abstract way the instantiation boilerplate. At least I hope everyone is keeping an open mind and is willing (and was prepared) to backtrack if something doesn't seem to be working.
w-m 3y ago
Every time the GIL removal is discussed, I notice a tidal wave of negativity in the comments, which is completely surprising to me. I suspect some critics primarily deal with thread-parallel programs that focus on asynchronous data sharing and task signaling. In languages like C++, handling these with threads and locks is tricky, and you're all but guaranteed to produce subtle race conditions. I'm totally with you on this – I wouldn't want to manage this in Python either. However, consider parallel data processing, a task I face so often in computer vision. In C++, I can slap a `pragma omp parallel for` onto a for loop processing a list of images/videos/meshes/csvs, increasing the speed of this section of the program by the number of CPU cores. I do this when the problem is well defined (one input, one output each), and it hasn't blown up in my face once. In Python, on the other hand, it feels cumbersome when spinning up worker processes for such tasks, not to mention harder to debug. For this kind of work, removing the GIL would be a massive relief. It's good that people are highlighting the issues. But to add my weight to the scale, the GIL is the main barrier in Python to stand in my way to speed up my programs. So I'm very happy that they're investing their resources to build a proper way of doing thread-parallelism.
- Eisenstein 3y ago
  I want to let you know that I agree with you 100% and face the same issues. It is baffling to me how this has not been solved by now, especially with Python's dominance in computer vision and machine learning.
- deleted 3y ago
  [deleted]
- dagw 3y ago
  It's worth remembering that the first serious gil removal patch was submitted against python 1.4, so it's not like people haven't been trying. There have also been (not 100% CPython compatible) python implementations without the GIL that people have could have used, but none of them got any traction. I suspect the negativity you're seeing comes mainly from two main sources. First people are terrified of another 2->3 situation. If removing the GIL ends up breaking any existing libraries or causing significant single-core performance degradations then that could do seriously damage to python and no one wants to go through that again. Secondly people who have been working on this for a long time get annoyed when people make removing the GIL sound easy and imply that the only reason it hasn't been done is because the developers are lazy, incompetent or haven't thought of it. Various people have been trying for literally decades, but so far no one has managed to come up with a solution that is both backwards compatible and performant. Everybody agrees that removing the GIL would be fantastic. However most of the core developers aren't willing to sacrifice backwards compatibility with existing code to get there.
molly0 3y ago
The best way to remove the GIL completely is switching language to Go, unfortunately this is not always possible.
FartyMcFarter 3y ago
From one of the linked threads: [1] > I might be good to remind the readers that the GIL removal has very little chance to break a Python-only codebase Is this actually true? I was under the impression that some multi-threaded Python code relies on some operations being implicitly thread safe due to the GIL. For example, adding an item to the same list from two concurrent threads is never going to corrupt the list, simply because the threads never do that operation in parallel (the GIL prevents the threads from running in parallel). If you remove the GIL, suddenly you'll punish this kind of code, just like C++ quickly punishes concurrent mutation of an std::vector. I'm not 100% sure of this, but I find this sentence a bit suspect. [1] https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional/22606/33 https://discuss.python.org/t/pep-703-making-the-global-inter...
- Znafon 3y ago
  Yes this is a thing was trivially protected by the GIL. There is the same thing with mutating the same map concurrently in Go that will panic for example. PEP 703 goes over this in the "Container Thread-Safety" (I think container here refers to the fact that the object has references to other objects, this is the things that already are special-cased in CPython to be managed specifically by the garbage collector): > This PEP proposes using per-object locks to provide many of the same protections that the GIL provides. For example, every list, dictionary, and set will have an associated lightweight lock. All operations that modify the object must hold the object’s lock. Most operations that read from the object should acquire the object’s lock as well; the few read operations that can proceed without holding a lock are described below. More information at https://peps.python.org/pep-0703/#container-thread-safety https://peps.python.org/pep-0703/#container-thread-safety
  - FartyMcFarter 3y ago
    Thank you! That makes sense, and it also explains why removing the GIL has a negative performance impact as discussed in other comments. Taking a lock every time a container is accessed is significant overhead, which is why languages like C++ don't make basic containers thread-safe.
tw1984 3y ago
Quit python 5 years ago, never looked back. If you want to write some glue code to put pieces together, golang is like 100x times better.
- adamckay 3y ago
  Great. And when there's a thread about Go I'm sure someone will tell us they don't use it and a different language is 100x better, too.
  - tw1984 3y ago
    That language is called C and I use/read/enjoy/teach/love/worship it everyday.
  - stonemetal12 3y ago
    Getting better is a process, you have to take it one step at a time. /s
amelius 3y ago
Is there any chance that CoPilot can help transform old-style C extensions into new style api code? (Since the transformation is probably going to be very similar for all those extensions).
samsquire 3y ago
Thank you to everyone involved in this project and thank you lwn for the article. I am a beginner in this space but my hobby is paralellism, coroutines, async and multithreading. I read about the subinterpreter approach and I think the subinterpreter approach causes me to think of actors. You destroy the reference in the source interpreter when you send data to the other interpreter, like message passing. But it should be a O(1) message passing and transfer of ref counting responsibilities. The interpreters would be separate so you wouldn't have to worry about a dictionary being updated in two threads because they never touch the same objects. After reading https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the... I am thinking about how to parallelise the creation of relationships. What's a good practice for parallelising behaviour? If I want to fan out a message to 20 million collections, how can I use paralellisation to make it faster?
hospitalJail 3y ago
A few questions: If single threaded performance is degraded, couldn't these people use an old version of python? With such incredible increases in multithreaded performance, I imagine this is basically infrastructure tier importance. Like the US government should be funding it. Would throwing a billion dollars at it, solve it in ~1 year? Or is this going to take 3 years regardless?
smcl 3y ago
I am quite impressed by how Sam Gross pushed for it. After he got a positive but non-committal response from the steering committee it would've been easy to just sit back and hope for some movement. The fact that he disagreed and pressured them is to his immense credit.
ed_blackburn 3y ago
As an application developer, I am curious about where this is leading/empowering. I hope to see something akin to the Project Loom approach in Java whereby suitable parallelisation abstractions are native, and one configures concrete implementations. Examples off the top of my head are: 1/ Light threads with default implementation of a scheduler provided by CPython and not dissimilar to Golang. Orthogonal to adjunct libraries or potential language features like channels. 2/ A no-op that uses the GIL can be default or even an interpreter switch, as discussed in the article. 3/ A stable ABI so one can replace the CPython implementation with an alternative. Written in Rust or whatever could be for hardware or a cloud vendor's environment. 4/Sub-interpreters, where parallelism pays to be far more coarse, could be Actor like architectures where spawned processes are new interpreters whose Actor System is a graph of interpreters. These envisioned use cases offer significant advantages, and the potential benefits are exciting and encouraging. The prospect of this progress is exhilarating, and I eagerly anticipate this endeavour's evolution. Thanks to all those painstakingly putting so much energy into this. Edit: spelling and formatting
wmichelin 3y ago
At my job we get pretty far just using things like einhorn to just run a bunch of similar python processes under the same port- this mostly fixes the single threaded performance problem.
kortex 3y ago
Very exciting. Sam Gross took a very bold move with his "I don't need a yes/no right now, but I do need to know what acceptance looks like, and this issue has been languishing." [0] That interaction could have gone a lot of different ways (especially as a neurospicy engineer myself and knowing the overall spice level in the computer world) but it sounds like that was the exact kick the council needed. It is still a long and winding path (double-digit engineer-years) to get a no-gil python, but at least there exists a proper path from the looks of it. The hardest part by far will be ensuring the correctness of all existing codebases. It's one thing to say you don't want a 2->3 repeat. It's another altogether if you were to claim no breaking changes, but fear of bugs resulted in folks avoiding the upgrade in practice. Making gil/no-gil even a compile-time switch will absolutely increase the maintenance cost. But I think in the long run, all this effort will be worth it, as I would claim the GIL is a lightning rod for python criticism. Just peruse any HN thread about python and parallelism to see what I mean. Maybe it's because it's the one thing folks can point directly to and say "this is why python isn't as fast as it could be" without understanding the decades of context. It's kind of the Final Boss of Chesterton's Fences in that regard. [0] https://github.com/python/steering-council/issues/188#issuecomment-1581534250 https://github.com/python/steering-council/issues/188#issuec...
- notconvinced 3y ago
  I'm not quite as optimistic as your take. The decision by the steering committee seems wishy washy to me. It's still possible for a multiyear effort to remove the GIL to get rejected even under these new guidelines.
qtieb 3y ago
This is very exciting and I'm thankful that they're actively avoiding another disruptive transition like the one from Python 2 to 3.
The_Colonel 3y ago
My private opinion, but I think removing GIL is a mistake. Not that many applications will benefit strongly from this, most will suffer a performance penalty. This will take years of attention/effort which could be spent more wisely. It kind of seems like Python suffers from insecurity / inferiority complex about GIL. I would instead go with the "JavaScript route" of fully embracing the single thread model. Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway. Being somewhat specialized and not supporting every use case isn't necessarily bad.
- klodolph 3y ago
  It may be too late to go with the JavaScript model.
  - The_Colonel 3y ago
    Why?
    - klodolph 3y ago
      There’s a lot of deployed Python code that uses multithreading. The genie is out of the bottle, so to speak. The JavaScript model is, basically, the sub-interpreter model. You have multiple independent heaps. Given independent heaps, you write single-threaded code and don't worry about locking, because no other thread can modify objects in your heap anyway, outside of special objects like SharedArrayBuffer. Maybe you could enable GIL-free, single-threaded, sub-interpreter Python as a compile-time option. But it would break such a large amount of code out there. It would be a very difficult transition, practically speaking.
- Znafon 3y ago
  The PEP goes into details about the motivations and why the single thread model has limitations: https://peps.python.org/pep-0703/#motivation https://peps.python.org/pep-0703/#motivation. > Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway People are already trying to use Python for parallel tasks. Forcing people to use another language does not help them much.
crabbone 3y ago
> If there's one lesson we've learned from the Python 2 to 3 transition, it's that it would have been very beneficial if Python 2 and 3 code could coexist in the same Python interpreter. We blew it that time, and it set us back by about a decade. A very characteristic thing for a core Python developer to say: it's stupid, baseless, but gets a standing ovation. The reason Python 3 wasn't catching up is because it offered no tangible benefits. Had it been, say, 10x faster than Python 2, then there would've been a reason. Had it offered sensible concurrency, there would've been a reason, had it offered some way to automatically verify programs... or a bunch of other useful things you might expect from a language, there would've been a reason to switch. But Python 3 was and still is a worthless change. It just gives you the ability to do the same things you could already do but in a slightly different way. If CPython project didn't refuse to support Python 2, it would be still alive and well. I've lived for a while with the language that had to support two versions at the same time: ActionScript. After AS3 was released, the SWF format and Flash player had changed to support executing both at the same time. The cases when both were used at the same time were very rare. They were never a part of an upgrade process -- usually it was because someone had to embed code they were under contractual obligation to execute (s.a. showing ads). Nobody partially upgraded their projects. Now, even though Python didn't provide a way to run two versions of the language at the same time, it wouldn't have been hard to do on your own, if you really wanted that: say modify the multiprocessing module to launch Python 2 from Python 3 or Python 3 from Python 2. Or create a native module embedding a Python interpreter of a different version. There are other ways as well... but nobody was doing that. Nobody needed that. There's no evidence anyone wanted to run both versions of Python at the same time, but, instead of facing the music and understanding that Python 3 wasn't wanted, these people will pet themselves on the back, tell each other how great their project is, and how it doesn't succeed for any other reason beside them being incompetent.
- nerdponx 3y ago
  The 2/3 split was intended to allow Python to get better in the future. Python 3.11 is tremendously better than 2.7, but you couldn't have got to that point without the 2/3 split first.

339 points by signa11 3y ago

What a great article from LWN. It was well-worth reading. As someone who was excited about the NoGIL from Sam Gross when it was first posted here, I think I'm beginning to change my mind after reading this article and reflecting on my own personal experiences. My experience is with writing backend systems in several different languages (including Python) at various volume/latency/throughput levels. I've basically worked on only two types of systems - 1. one that exposes some sort of an endpoint to the network - it accepts requests of some kind, does computation and other network requests and sends response of some kind (including long polling, ws etc). 2. reads a message from a "queue" (could be database, could be based on polling another api etc) and does computation/network calls and basically sends it to other queues. Nothing else. Huge variance in specific requirements, but that's it. For the first type of system, latency matters more. For the second system, throughput matters more. For the first type of system, I want to be able to spin up threads in response to requests, without worrying that an endpoint is too computationally heavy and might block others. I want to be able to share connections to databases in a shared pool. NoGIL would be useful here. For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines. I would absolutely be disappointed if NoGIL meant compromising on the quality of of the second type of system here. In practice, most of my mental bandwidth today goes towards making the second type of system better.

YeBanKo 3y ago
To take advantage of NoGIL you don’t necessarily need to use parallelism directly. But let’s say your web server or async task executor can be more efficient at sharing context between threads.
VagabundoP 3y ago
As a hobbyist who uses python I don't think I'll be directly using concurrency in my code, but I'm betting that over time the standard library and popular external libraries will. And that will raise everyone's code.
scarygliders 3y ago
> For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines. For myself, the prospect of no-gil is interesting, in that something like my Captain's Log application [0] can be free from it; for example, I currently use a QThread to implement a JournalParser, which is basically the program's "engine" - the parser constantly reads in game events from a player journal file generated by the game Elite: Dangerous (and Odyssey), and depending on the particular event, fires off a related custom QSignal, which is then processed by whichever slot (receiving function) is listening for a given Signal. There are other places in that application where no GIL might be quite handy. In other words, I can see where having no GIL can be useful for GUI applications like mine. [0] https://captainslog.scarygliders.net/captains-log-2/ https://captainslog.scarygliders.net/captains-log-2/
- radarsat1 3y ago
  Your JournalParser sounds like it could be implemented using normal Python threads or by an asyncio event loop without much performance problems. If I understand all it's doing is watching for events and posting a signal somewhere, so it doesn't sound like the kind of application that is CPU-bound.
  - scarygliders 3y ago
    I could. But I 100% take full advantage of Qt's signal and slot mechanism. Also, in many ways, GUI applications written in Python are not so much CPU bound, but Python GIL bound. If you're writing a Python/Qt application, you have to take great care to ensure your GUI doesn't freeze when your program is performing, say, many database inserts; if you have some naive loop which performs some given operation, your nice Qt GUI will freeze right up until that operation is complete. Right now the solution is to perform such operations in, say, a QThread, and use Qt's signal/slot feature to blat a progress "report" to a handler in the `main` Python loop. So back to what I said - no-GIL is looking quite interesting to me. Whether or not Qt can take advantage of such will be a different matter.
spacechild1 3y ago
The GIL is a bottleneck in applications that are CPU bound, e.g. machine learning, so naturally the NoGIL project is not that interesting to people writing server applications. Of course, one may argue that you probably should not write CPU bound programs in Python in the first place, but that's another story :)
- jillesvangurp 3y ago
  A lot of Java server based applications are multi threaded, not CPU bound, and tend to do a lot of things that generally can't work in python because of the GIL. It's too simplistic to think of this as something only of interest for CPU bound stuff. A lot of what Java applications do is of course the thread per connection style processing that older java applications still do (more modern ones would use non blocking IO and green threads). But there are also background threads doing useful work or more complex requests that fork off asynchronous work across multiple CPUs and then aggregate the results back as the response. Java apps tend to have vastly more threads than CPU cores. The exception is when things are CPU bound; then you want to minimize the context switching and end up with an number that is close to the number of CPU cores. The GIL is not about the CPU but about enabling those kinds of things. With the current GIL in place it's very simple: as soon as you hit the global lock, everything stops until it is released. It doesn't matter how many CPU cores you have, they'll be idling while one of them holds the lock. There's barely any point in even trying to do that with the GIL in place. Forget about sharing data between threads. Mostly that's done via queues or databases in python. Removing the GIL will revolutionize a few things in key use cases for python: - data processing & ETL - event driven server systems - machine learning and data science systems They can all benefit from this and that's the reason a lot of people are pushing for this. The short term performance losses are not inherent to removing the GIL but just a necessary evil while the python developers deal with fixing the bottlenecks and a few decades worth of technical debt.
  - spacechild1 3y ago
    I/O functions (may) internally release the GIL. If the GIL becomes a bottleneck, you are not I/O bound by definition. However, you are certainly right that not all server applications are I/O bound. I was a bit sloppy there.
mirsadm 3y ago
>For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines. Interestingly I'm working on something like this right now and do have large shared resources which meant I had to abandon using a multiprocess strategy. I don't see why it would be confusing though, provided the shared resources are read only.
- nullspace 3y ago
  For such applications, isn’t the parallelism count usually static and limited? I think there will be good benefits for distributed system frameworks for python. Id agree. A couple of years ago, I implemented in process parallelism for a system I was maintaining at $JOB. I was happy the system was in Go and not Python. But it was an exception to the rule in my experience.

w10-1 3y ago

I can't imagine a more difficult test case for open source. The decision is entirely reasonable, to move forward in test mode, treat multiple interpreters as an interim experiment, but target being concurrent. The constraint of running old and new code in the same VM is a tall one. Surprisingly, LWN's summary says nothing about testing, which is largely unsolved and could lead to releases with unknown but serious bugs. Microsoft, Facebook/Meta, and Conda have stepped up with resources and a super-majority of core contributors want to move forward, but it's unclear what happens if things get hard and more people are needed. Meanwhile a crazy number of projects in academia and industry from web sites to big data to AI depend on Python. The potential to export costs to python developers might be measured in percentage of GDP. It doesn't sound like the problems are even known yet. It might be fairer to commit to the Faster CPython approach of knowable improvement over the next 3+ years, but have the concurrent Python promotors do more than just prototype. They should analyze the kinds of problems that could present, how they could be detected, and what can be done about it. This should be reviewed by people with backgrounds in proving concurrency guarantees. Then the questions can be fairly presented to the steering committee, when the unknowns are at least identified. There's not a lot of program-management scale decision-making in open-source, but most have been relatively simple questions of driving the market in a given direction (apache, eclipse, linux). This has real technical unknowns. And I can't help feeling they should also address inter-language ABI's at the same time. A big issue is matching the expected execution model of C. Java's foreign-function and memory interfaces have been incubating for many years, and Swift is also getting better at wrapping C and C++, but FFI's are notoriously (and likely unnecessarily) difficult.

michaelt 3y ago
> The potential to export costs to python developers might be measured in percentage of GDP. Of course, that's also true of the benefits.

BiteCode_dev 3y ago

The article is well written, and a good history of the all ordeal. But please note it gives more weight to the "against the GIL" side of the story, and all that can go wrong. It doesn't highlight enough the other side of the coin: - Sam's work is very high quality, and he brought with the no-gil some unrelated perf improvements so that people don't feel like they loose perf too much. - Sam played the open source game perfectly, and is incredibly patient given what he is bringing on the able and how slow and flaccid the steering council reaction was (without the community pushing on it, it would still be collecting dust). - Sub interpreters have yet to demonstrate any usefulness at all in Python. In fact, any serious metrics at all. This is the first attempt to be that well defined, and measured. - The community feedback shown a great interest in this particular project. - The steering council did conclude "We intend to accept PEP 703, although we’re still working on the acceptance details." I'm not a no-gil enthusiast. I would be fine with have it never be removed, and I think we should try sub-interpreter first. But what's fair is fair.

TylerE 3y ago
What I’d liked to see is the performance patches without the no-Gil part.
- Znafon 3y ago
  The performance patches have already been merged and there is ongoing working in the Faster CPython project.
- formerly_proven 3y ago
  You're already using a bunch of those in recent Python versions.
Znafon 3y ago
> I would be fine with have it never be removed, and I think we should try sub-interpreter first. There is a lot of work going on with sub-interpreters and the per-interpreter GIL is shipped in Python 3.12. The results are very impressive: https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/ https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/, as good as one could have hoped I think. It seems to me like the work on sub-interpreters will continue in parallel ;) to the work on free-threading. Sub-interpreters and no-GIL have different use-cases though.

zarzavat 3y ago

It’s more important to prioritize single-threaded performance because it’s much harder to improve by throwing money at the problem. With multithreaded performance, you can just add another core to (more than) offset whatever overheads there are from using process-based parallelism. I think that this entire GIL vs No-GIL dichotomy is misguided. The biggest problem people have with multiprocessing is that you can’t share memory. So add virtual processes with an explicit mechanism for memory sharing. Then you can keep all of your single-threaded optimizations like refcounting without barriers because the objects for one thread will stay in that thread.

Galanwe 3y ago
Agree 100%. If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. Fixing _that_ should be the objective. What Python need is better support for placing native instances in shared memory.
- Znafon 3y ago
  > If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. > The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. The PEP goes into the motivation behind this work, and using multiple process does not magically solves all the issues: > Multiprocessing, with communication via shared memory or UNIX sockets, adds much complexity and in effect rules out interacting with CUDA from different workers, severely restricting the design space. > I reimplemented parts of HMMER, a standard method for multiple-sequence alignment. I chose this method because it stresses both single-thread performance (scoring) and multi-threaded performance (searching a database of sequences). The GIL became the bottleneck when using only eight threads. This is a method where the current popular implementations rely on 64 or even 128 threads per process. I tried moving to subprocesses but was blocked by the prohibitive IPC costs. > NumPy does release the GIL in its inner loops (which do the heavy lifting), but that is not nearly enough. NumPy doesn’t offer a solution to utilize all CPU cores of a single machine well, and instead leaves that to Dask and other multiprocessing solutions. Those aren’t very efficient and are also more clumsy to use. That clumsiness comes mainly in the extra abstractions and layers the users need to concern themselves with when using, e.g., dask.array which wraps numpy.ndarray. It also shows up in oversubscription issues that the user must explicitly be aware of and manage via either environment variables or a third package, threadpoolctl. The main reason is that NumPy calls into BLAS for linear algebra - and those calls it has no control over, they do use all cores by default via either pthreads or OpenMP. and it discusses the alternatives at https://peps.python.org/pep-0703/#alternatives https://peps.python.org/pep-0703/#alternatives.

Mayzie 3y ago

I understand the desire for multi-threaded CPython scripts, but why not do a PHP5 -> PHP7 and focus on single-thread performance more, particularly around implementing a JIT? We know from PyPy (and other JIT-enabled languages like NodeJS) that performance improvements can be significant. I feel like that should be the primary focus for speedups, and have always wondered why the idea has never been more popular. Has a CPython JIT been ruled out for some reason I'm not familiar with?

turndown 3y ago
You should look into the copy & patch efforts underway for Python[0]; an actual JIT will probably never exist but I think c&p has a shot of being mainlined in the next few years, such that Python could dynamically choose to either run the interpreter or a c&p option. 0: https://github.com/faster-cpython/ideas/issues/588 https://github.com/faster-cpython/ideas/issues/588
dagw 3y ago
Has a CPython JIT been ruled out for some reason There have been several attempts at integrating a JIT into CPython. Google made a stab at it back around version 2.6 and there was a second attempt around the 2->3 change to build a JIT on top of LLVM. However none of these projects produced results that the core developers felt were good enough across the board and had problems with backwards compatibility and lot cross platform LLVM issues (as it was a pretty new project) so they were dropped after Google stopped funding the project. That being said there is still work being done and people are hoping for at least some JITing in upcoming python. Pyston is project being worked on Guido Van Possum himself and both Microsoft Instagram have their own JITed versions of Python they use internally. Some initial parts of Pyston are even scheduled for being included in python 3.12 or 3.13
josefx 3y ago
PyPy breaks compatibility with CPython in significant ways, as far as I understand the current no-gil proposal only got through because it has an automated fallback for modules that are not no-gil compatible.
- 3y ago

VagabundoP 3y ago

Excellent write up as usual from LWN. I love the Python community. It's really a leading light for open source software. And it shows what transparency and good governance can achieve. Although I appreciate the engineering hours that Meta, Microsoft and others give, its pretty miserable still compared to the value that the whole tech industry (and beyond with data science) extract from Python and other open source software.

stevesimmons 3y ago
We can all help to change that, within our own organisations! I did my bit at JPMorgan 8 years ago, convincing the tech leadership team to sponsor PyCon UK, plus a recruiting stand and supporting a group of junior developers from all JPMorgan's UK locations to attend. I left JPM 5 years ago now, and they are still PyCon UK's headline sponsor. Compared with the enormous benefit we got from Python and its open source ecosystem, it was a totally negligible cost.
lrtzyv 3y ago
They pretend to be transparent. All actual decisions are made in backrooms. The mailing lists are censored and the inner circle cannot be criticized. Real contributors are exploited by those who work for the right corporations, do very little and go for any clerical position of power. Do not be misled by the LWN articles, which are very kind and always namedrop the deciders. It is selective reporting.
- nojonestownpls 3y ago
  > The mailing lists are censored and the inner circle cannot be criticized. Well, HN isn't censored in this regard (unless you believe their influence extends here too). What specific criticisms do you have against "the inner circle"? And what evidence can you show for them?

TekMol 3y ago

There was a question from Shannon about ""what people think is a acceptable slowdown for single-threaded code"" ... he had estimated an impact "in the 15-20% range" Horror! To me, the acceptable slowdown is exactly zero. I can already use multiple cores by running multiple Python processes. Any slowdown of single-process performance would be a terrible step backwards. Python is already slow. It should look at PHP and see what it can take out of their book. PHP 7, which had no JIT, was already about 6x faster than Python. PHP 8 is even faster for some workloads. I'm not sure if the overhead of a JIT makes sense though. But PHP 7 is a good place to look for a performance benchmark.

deleted 3y ago
[deleted]
dagw 3y ago
To me, the acceptable slowdown is exactly zero. This has been the reason most of the previous GIL removal projects failed. GvR had a strict zero slowdown in single-thread performance rule for any nogil patches and none of them managed that. I wonder if the core team has decided to abandon that rule now that GvR is gone?
- alecmg 3y ago
  I saw comments that if introduced together with Faster CPython single thread performance optimisations, nogil would end up no slower than previous version. So it is best time to implement
ohgodplsno 3y ago
>I can already use multiple cores by running multiple Python processes. Because you have terribly simple processes that don't need to do synchronization. Even a simple parallel map would have Python scream and kick, because the GIL prevents it from having any kind of reasonable performance, and your multiprocess can't handle that unless they all write to files.
pkolaczk 3y ago
> Horror! To me, the acceptable slowdown is exactly zero. If you care about single % of performance, why are you using Python in the first place? You're likely losing 100x to more performant languages already.

globular-toast 3y ago

Multi-threading gets you programs that are, at the very most, 8x faster. Is it really worth it? If this makes it even slightly more difficult to develop Python for normal people then it will get forked, and that sucks. If it can be done without any impact then great, but I'm still not sure it's worth it.

josefx 3y ago
> at the very most, 8x faster. Where does that number come from? My current desktop has 32 logical cores and it has been a few years since I bought that one.
- alecmg 3y ago
  maybe its a reference to Amdahl's law? If you can parallelise 90% of your code, you get only 10x improvement even on infinite number of cores
  - josefx 3y ago
    You mix up time and code. You don't have to parallelize 100% of your code to scale with cores, you only have to parallelize the code that your software spends most of its time executing and that is often significantly less than 90%.
usrbinbash 3y ago
> at the very most, 8x faster. My 64 core server setup says otherwise. > Is it really worth it? Yes, very much so. > If this makes it even slightly more difficult to develop Python for normal people It won't. Pure python code will very likely not notice a difference.

coldtea 3y ago

>There was a question from Shannon about "what people think is a acceptable slowdown for single-threaded code". To a large extent, that question went unanswered in the thread, but he had estimated an impact "in the 15-20% range, but it could be more, depending on the impact on PEP 659". So, let me get this straight, some of those working in the project to "make CPython faster", think it's acceptable to overnight make most existing Python code 15-20% slower? I'd say max 5% and that if the gil removal was a benefit to other optimizations going forward (they say the opposite: the change with complicate and stall their other optimizations). Meanwhile, Shannon had a fundraising proposal for him personally "speeding up CPython" 5x back in 2020. Now it's a whole team working on speeding up CPython with much larger corporate support, and it seems their targets are quite smaller?

tda 3y ago
But consider that in perhaps 99% of current python code high performance (of the pure python code) is not the primary concern. And now for the times when performance does matter you can get a 20x (?) speedup without rewriting in another language.
- coldtea 3y ago
  I'd expect in many those cases people care for performance, they already use multiprocess (so the impact from a 20% hit GIL-free CPython would actually be a slowdown). And for those that do not use multiprocess, rewriting their currently single threaded Python code for no-gil shared-concurrency will indeed bring a speedup, but would still be a significant rewrite. Also there are people using Python for things like web serving, where performance might have been secondary to convenience when they picked Python, but still a sudden 15-20% slowdown will seriously impact their server budgets (or ability to update versions)...
- School-Cotton 3y ago
  > But consider that in perhaps 99% of current python code high performance (of the pure python code) is not the primary concern. Then why do we care about GIL removal at all?
3y ago

londons_explore 3y ago

I want to see python with automatic and transparent concurrency. Ie. The programmer writes: for file in glob.glob('*.jpg'): data.append(load_file(file)) Then python itself parallelizes that work. It should do that by identifying loops of pure functions and pushing them off to other threads. One should also be able to mark a function as pure. Python itself would be responsible for making everything appear as if the code ran serially - for example by buffering/reordering log statements. Rationale: Python is designed to be easy and simple. Concurrency in general is not that. Python should do only do concurrency in a way that can be made easy and simple.

lmeyerov 3y ago
Not sure if intentional: That's a super frustrating program for a traditional automatic parallelization compiler, e.g., polyhedral: * glob is an i/o operation: impure * load_file is an i/o operation: impure * data.append serializes part of the execution order --- Interestingly, GPT4 does pretty ok: files = glob.glob('*.jpg') with ThreadPoolExecutor() as executor: data = list(executor.map(load_file, files)) --- Separately, if just going for concurrency for i/o, async/await is pretty amazing: tasks = [load_file(file) for file in glob.glob('*.jpg')] data = await asyncio.gather(*tasks) A lot less program transformation needed as it's essentially sugar over promises, and avoids much of the need to restructure code beyond the coloring problem. So I'm not as much about the case for a compiler here..
- londons_explore 3y ago
  > not sure if intentional. It sure was! I believe any automatic parallelization should be able to deal with all these things. With a combination of opportunistic execution (ie. guess which bits can be parallelized, and roll back if wrong), and clever heuristics to avoid rollbacks, it should be pretty do-able. CPU manufacturers do similar stuff at a smaller scale with parallel execution of single threaded code with Tomasulo's algorithm, and they get massive gains compared to non-superscalar CPU's. They even do similar stuff at medium scale with Hardware Lock Eliding.
- starlevel003 3y ago

mihaic 3y ago

Great exposition of the situation! I was in the no-GIL camp until a few years ago, until I started thinking of all the implications: I have tons of code that modifies global maps from all over the place, and I feel relieved to not manage threads like in C/C++. What I hope the solution will end up emerging would be subinterpreters that are more intuitive to use. Honestly, I don't want to pass in source code, but rather a function or module entrypoint and have the subinterpreter abstract way the instantiation boilerplate. At least I hope everyone is keeping an open mind and is willing (and was prepared) to backtrack if something doesn't seem to be working.

w-m 3y ago

Every time the GIL removal is discussed, I notice a tidal wave of negativity in the comments, which is completely surprising to me. I suspect some critics primarily deal with thread-parallel programs that focus on asynchronous data sharing and task signaling. In languages like C++, handling these with threads and locks is tricky, and you're all but guaranteed to produce subtle race conditions. I'm totally with you on this – I wouldn't want to manage this in Python either. However, consider parallel data processing, a task I face so often in computer vision. In C++, I can slap a `pragma omp parallel for` onto a for loop processing a list of images/videos/meshes/csvs, increasing the speed of this section of the program by the number of CPU cores. I do this when the problem is well defined (one input, one output each), and it hasn't blown up in my face once. In Python, on the other hand, it feels cumbersome when spinning up worker processes for such tasks, not to mention harder to debug. For this kind of work, removing the GIL would be a massive relief. It's good that people are highlighting the issues. But to add my weight to the scale, the GIL is the main barrier in Python to stand in my way to speed up my programs. So I'm very happy that they're investing their resources to build a proper way of doing thread-parallelism.

Eisenstein 3y ago
I want to let you know that I agree with you 100% and face the same issues. It is baffling to me how this has not been solved by now, especially with Python's dominance in computer vision and machine learning.
deleted 3y ago
[deleted]
dagw 3y ago
It's worth remembering that the first serious gil removal patch was submitted against python 1.4, so it's not like people haven't been trying. There have also been (not 100% CPython compatible) python implementations without the GIL that people have could have used, but none of them got any traction. I suspect the negativity you're seeing comes mainly from two main sources. First people are terrified of another 2->3 situation. If removing the GIL ends up breaking any existing libraries or causing significant single-core performance degradations then that could do seriously damage to python and no one wants to go through that again. Secondly people who have been working on this for a long time get annoyed when people make removing the GIL sound easy and imply that the only reason it hasn't been done is because the developers are lazy, incompetent or haven't thought of it. Various people have been trying for literally decades, but so far no one has managed to come up with a solution that is both backwards compatible and performant. Everybody agrees that removing the GIL would be fantastic. However most of the core developers aren't willing to sacrifice backwards compatibility with existing code to get there.

molly0 3y ago

The best way to remove the GIL completely is switching language to Go, unfortunately this is not always possible.

FartyMcFarter 3y ago

From one of the linked threads: [1] > I might be good to remind the readers that the GIL removal has very little chance to break a Python-only codebase Is this actually true? I was under the impression that some multi-threaded Python code relies on some operations being implicitly thread safe due to the GIL. For example, adding an item to the same list from two concurrent threads is never going to corrupt the list, simply because the threads never do that operation in parallel (the GIL prevents the threads from running in parallel). If you remove the GIL, suddenly you'll punish this kind of code, just like C++ quickly punishes concurrent mutation of an std::vector. I'm not 100% sure of this, but I find this sentence a bit suspect. [1] https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional/22606/33 https://discuss.python.org/t/pep-703-making-the-global-inter...

Znafon 3y ago
Yes this is a thing was trivially protected by the GIL. There is the same thing with mutating the same map concurrently in Go that will panic for example. PEP 703 goes over this in the "Container Thread-Safety" (I think container here refers to the fact that the object has references to other objects, this is the things that already are special-cased in CPython to be managed specifically by the garbage collector): > This PEP proposes using per-object locks to provide many of the same protections that the GIL provides. For example, every list, dictionary, and set will have an associated lightweight lock. All operations that modify the object must hold the object’s lock. Most operations that read from the object should acquire the object’s lock as well; the few read operations that can proceed without holding a lock are described below. More information at https://peps.python.org/pep-0703/#container-thread-safety https://peps.python.org/pep-0703/#container-thread-safety
- FartyMcFarter 3y ago
  Thank you! That makes sense, and it also explains why removing the GIL has a negative performance impact as discussed in other comments. Taking a lock every time a container is accessed is significant overhead, which is why languages like C++ don't make basic containers thread-safe.

tw1984 3y ago

Quit python 5 years ago, never looked back. If you want to write some glue code to put pieces together, golang is like 100x times better.

adamckay 3y ago
Great. And when there's a thread about Go I'm sure someone will tell us they don't use it and a different language is 100x better, too.
- tw1984 3y ago
  That language is called C and I use/read/enjoy/teach/love/worship it everyday.
- stonemetal12 3y ago
  Getting better is a process, you have to take it one step at a time. /s

amelius 3y ago

Is there any chance that CoPilot can help transform old-style C extensions into new style api code? (Since the transformation is probably going to be very similar for all those extensions).

samsquire 3y ago

Thank you to everyone involved in this project and thank you lwn for the article. I am a beginner in this space but my hobby is paralellism, coroutines, async and multithreading. I read about the subinterpreter approach and I think the subinterpreter approach causes me to think of actors. You destroy the reference in the source interpreter when you send data to the other interpreter, like message passing. But it should be a O(1) message passing and transfer of ref counting responsibilities. The interpreters would be separate so you wouldn't have to worry about a dictionary being updated in two threads because they never touch the same objects. After reading https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the... I am thinking about how to parallelise the creation of relationships. What's a good practice for parallelising behaviour? If I want to fan out a message to 20 million collections, how can I use paralellisation to make it faster?

hospitalJail 3y ago

A few questions: If single threaded performance is degraded, couldn't these people use an old version of python? With such incredible increases in multithreaded performance, I imagine this is basically infrastructure tier importance. Like the US government should be funding it. Would throwing a billion dollars at it, solve it in ~1 year? Or is this going to take 3 years regardless?

smcl 3y ago

I am quite impressed by how Sam Gross pushed for it. After he got a positive but non-committal response from the steering committee it would've been easy to just sit back and hope for some movement. The fact that he disagreed and pressured them is to his immense credit.

ed_blackburn 3y ago

As an application developer, I am curious about where this is leading/empowering. I hope to see something akin to the Project Loom approach in Java whereby suitable parallelisation abstractions are native, and one configures concrete implementations. Examples off the top of my head are: 1/ Light threads with default implementation of a scheduler provided by CPython and not dissimilar to Golang. Orthogonal to adjunct libraries or potential language features like channels. 2/ A no-op that uses the GIL can be default or even an interpreter switch, as discussed in the article. 3/ A stable ABI so one can replace the CPython implementation with an alternative. Written in Rust or whatever could be for hardware or a cloud vendor's environment. 4/Sub-interpreters, where parallelism pays to be far more coarse, could be Actor like architectures where spawned processes are new interpreters whose Actor System is a graph of interpreters. These envisioned use cases offer significant advantages, and the potential benefits are exciting and encouraging. The prospect of this progress is exhilarating, and I eagerly anticipate this endeavour's evolution. Thanks to all those painstakingly putting so much energy into this. Edit: spelling and formatting

wmichelin 3y ago

At my job we get pretty far just using things like einhorn to just run a bunch of similar python processes under the same port- this mostly fixes the single threaded performance problem.

kortex 3y ago

Very exciting. Sam Gross took a very bold move with his "I don't need a yes/no right now, but I do need to know what acceptance looks like, and this issue has been languishing." [0] That interaction could have gone a lot of different ways (especially as a neurospicy engineer myself and knowing the overall spice level in the computer world) but it sounds like that was the exact kick the council needed. It is still a long and winding path (double-digit engineer-years) to get a no-gil python, but at least there exists a proper path from the looks of it. The hardest part by far will be ensuring the correctness of all existing codebases. It's one thing to say you don't want a 2->3 repeat. It's another altogether if you were to claim no breaking changes, but fear of bugs resulted in folks avoiding the upgrade in practice. Making gil/no-gil even a compile-time switch will absolutely increase the maintenance cost. But I think in the long run, all this effort will be worth it, as I would claim the GIL is a lightning rod for python criticism. Just peruse any HN thread about python and parallelism to see what I mean. Maybe it's because it's the one thing folks can point directly to and say "this is why python isn't as fast as it could be" without understanding the decades of context. It's kind of the Final Boss of Chesterton's Fences in that regard. [0] https://github.com/python/steering-council/issues/188#issuecomment-1581534250 https://github.com/python/steering-council/issues/188#issuec...

notconvinced 3y ago
I'm not quite as optimistic as your take. The decision by the steering committee seems wishy washy to me. It's still possible for a multiyear effort to remove the GIL to get rejected even under these new guidelines.

qtieb 3y ago

This is very exciting and I'm thankful that they're actively avoiding another disruptive transition like the one from Python 2 to 3.

The_Colonel 3y ago

My private opinion, but I think removing GIL is a mistake. Not that many applications will benefit strongly from this, most will suffer a performance penalty. This will take years of attention/effort which could be spent more wisely. It kind of seems like Python suffers from insecurity / inferiority complex about GIL. I would instead go with the "JavaScript route" of fully embracing the single thread model. Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway. Being somewhat specialized and not supporting every use case isn't necessarily bad.

klodolph 3y ago
It may be too late to go with the JavaScript model.
- The_Colonel 3y ago
  Why?
  - klodolph 3y ago
    There’s a lot of deployed Python code that uses multithreading. The genie is out of the bottle, so to speak. The JavaScript model is, basically, the sub-interpreter model. You have multiple independent heaps. Given independent heaps, you write single-threaded code and don't worry about locking, because no other thread can modify objects in your heap anyway, outside of special objects like SharedArrayBuffer. Maybe you could enable GIL-free, single-threaded, sub-interpreter Python as a compile-time option. But it would break such a large amount of code out there. It would be a very difficult transition, practically speaking.
Znafon 3y ago
The PEP goes into details about the motivations and why the single thread model has limitations: https://peps.python.org/pep-0703/#motivation https://peps.python.org/pep-0703/#motivation. > Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway People are already trying to use Python for parallel tasks. Forcing people to use another language does not help them much.

crabbone 3y ago

> If there's one lesson we've learned from the Python 2 to 3 transition, it's that it would have been very beneficial if Python 2 and 3 code could coexist in the same Python interpreter. We blew it that time, and it set us back by about a decade. A very characteristic thing for a core Python developer to say: it's stupid, baseless, but gets a standing ovation. The reason Python 3 wasn't catching up is because it offered no tangible benefits. Had it been, say, 10x faster than Python 2, then there would've been a reason. Had it offered sensible concurrency, there would've been a reason, had it offered some way to automatically verify programs... or a bunch of other useful things you might expect from a language, there would've been a reason to switch. But Python 3 was and still is a worthless change. It just gives you the ability to do the same things you could already do but in a slightly different way. If CPython project didn't refuse to support Python 2, it would be still alive and well. I've lived for a while with the language that had to support two versions at the same time: ActionScript. After AS3 was released, the SWF format and Flash player had changed to support executing both at the same time. The cases when both were used at the same time were very rare. They were never a part of an upgrade process -- usually it was because someone had to embed code they were under contractual obligation to execute (s.a. showing ads). Nobody partially upgraded their projects. Now, even though Python didn't provide a way to run two versions of the language at the same time, it wouldn't have been hard to do on your own, if you really wanted that: say modify the multiprocessing module to launch Python 2 from Python 3 or Python 3 from Python 2. Or create a native module embedding a Python interpreter of a different version. There are other ways as well... but nobody was doing that. Nobody needed that. There's no evidence anyone wanted to run both versions of Python at the same time, but, instead of facing the music and understanding that Python 3 wasn't wanted, these people will pet themselves on the back, tell each other how great their project is, and how it doesn't succeed for any other reason beside them being incompetent.

nerdponx 3y ago
The 2/3 split was intended to allow Python to get better in the future. Python 3.11 is tremendously better than 2.7, but you couldn't have got to that point without the 2/3 split first.