16 ms·

The "Faster Python" team are doing a fantastic job, it's incredible to see the progress they are making. However, there is also a bit of a struggle going on be

The "Faster Python" team are doing a fantastic job, it's incredible to see the progress they are making. However, there is also a bit of a struggle going on between them and the project to remove the GIL (global interpreter lock) from CPython. There is going to be a performance impact on single threaded code if the "no GIL" project is merged, something in the region of 10%. It seems that the faster Python devs are pushing back against that as it impacts their own goals. Their argument is that the "sub interpreters" they are adding (each with its own GIL) will fulfil the same use cases of multithreaded code without a GIL, but they still have the overhead of encoding and passing data in the same way you have to with sub processors. There is also the argument that it could "divide the community" as some C extensions may not be ported to the new ABI that the no GIL project will result in. However again I'm unconvinced by that, the Python community has been through worse (Python 3) and even asyncIO completely divides the community now. It's somewhat unfortunate that this internal battle is happening, both projects are incredible and will push the language forward. Once the GIL has been removed it opens us all sorts of interesting opportunities for new concurrency APIs that could enable making concurrent code much easer to write. My observation is that the Faster Python team are better placed politicly, they have GVR on the team, whereas No GIL is being proposed by an "outsider". It just smells a little of NIH syndrome.

bratao 3y ago
Yeah, they clearly stated that here https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/118 https://discuss.python.org/t/pep-703-making-the-global-inter... I really wish that GIL go away. It is better to pay this price now, multi-threading is the future.
- hospitalJail 3y ago
  I'm so conflicted. ~40 of the programs I am responsible for are single threaded. They were relatively quick to develop and were made by Electrical Engineers rather than career/degreed programmers. 2 programs use multithreading, I had to do that. The learning curve was not a huge deal, but the development time adds at least hours. In my case days(due to testing). I imagine its too hard to have an optional flag at the start of each program that can let the user decide?
  - birdyrooster 3y ago
    The problem is that Python types are not thread safe so you have to jump through more hoops to have safe parallelization in Python. These changes would make it so that writing multithreaded code will be much easier it seems like.
  - maleldil 3y ago
    > I imagine its too hard to have an optional flag at the start of each program that can let the user decide? Adding nogil would mean deep changes to the interpreter. I imagine maintaining both versions would be almost like forking the project.
  - coldtea 3y ago
    >but the development time adds at least hours So? some "hours of development time" is nothing.
    - sfink 3y ago
      True, and as you get better at it, those hours will be fewer. The weeks of debugging never go away, though. (Or at least, not as long as you're using shared state, but that's really the only thing under consideration here.)
    - hospitalJail 3y ago
      I'm mostly with you. I think this probably affects part time/newbie programmers more. A few hours of dev time is 1 or 2 nights of work.
  - dragonwriter 3y ago
    > I imagine its too hard to have an optional flag at the start of each program that can let the user decide? The actual next-step proposal toward no-gil is GIL as a build-time flag (which isn’t quite the same as a runtime flag, but not too far off, either.) https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
  - nunuvit 3y ago
    It's not like CPython is depriving anyone of GIL-free Python. IronPython and Jython are GIL-free. You can have it yesterday. CPython maintains Python's original purpose as a glue language. You know how much worse CPython would be for that use-case without the GIL. GVR has said he'll support removing the GIL as long as there's no performance hit. Otherwise, it's simply asking for a fundamental change of direction and too much sacrifice from everyone who needs CPython specifically, just to improve the productivity of people who could use a different Python.
- sgt 3y ago
  Not for Python, I feel. In sheer volume, the vast majority of my Python programs are single-threaded. I want my programs to be very quick in runtime when they run. Those that are multi-threaded are seeing minor to medium load. If expecting extreme load (like Twitter scale), then Python is usually not the answer (rather go to a statically typed language like Java, Go, Rust etc).
  - gpderetta 3y ago
    > In sheer volume, the vast majority of my Python programs are single-threaded obviously if multithreading is near-useless in python, very few programs will take advantage of it.
    - zo1 3y ago
      Likewise I can say: Obviously we still have the GIL because very few people want it removed.
      - gpderetta 3y ago
        Well, yes, that's pretty much obvious: the majority of python programmers do not care, those that cared have moved on as it is easier than changing the language. And I'll be first to admit that improving single threaded performance has an higher priority than removing the GIL.
  - AlphaSite 3y ago
    Is single threaded perf is important, you’ve already lost by using python. You’re only ever going to get, ok-ish performance or slightly more ok-ish
  - gcbirzan 3y ago
    > Not for Python, I feel. In sheer volume, the vast majority of my Python programs are single-threaded. Yes, they are single threaded, because using multiple threads brings very little benefit in most cases... > If expecting extreme load (like Twitter scale), then Python is usually not the answer (rather go to a statically typed language like Java, Go, Rust etc). So that means we shouldn't get any performance improvements, because there are faster languages out there?
- amelius 3y ago
  It's relatively simple to make the GIL go away: just compile to some VM that has a good concurrent garbage collector would be one approach. Yes, this will break some assumptions here and there, but not too difficult to overcome especially if you bump the version number to Python 4. However, that leaves a lot of C code that you can't talk to anymore because the C code requires the old Python FFI. I think this is where the main problem lies.
  - notatallshaw 3y ago
    > It's relatively simple to make the GIL go away: just compile to some VM that has a good concurrent garbage collector would be one approach Sure, if you don't mind paying a 50-90% performance impact on single threaded performance or completely abandon C-API compatibility and have C extensions start from scratch then there are simple approaches. If you look at any example in the past to remove the GIL you would see that keeping these two requirements of not having terrible single threaded performance and not having almost a completely new C-API is actually very complex and takes a lot of expertise to implement.
    - saltminer 3y ago
      This might be a dumb question, but why would removing the GIL break FFI? Is it just that existing no-GIL implementations/proposals have discarded/ignored it, or is there a fundamental requirement, e.g. C programs unavoidably interact directly with the GIL? (In which case, couldn't a "legacy FFI" wrapper be created?) I know that the C-API is only stable between minor releases [0] compiled in the same manner [1], so it's not like the ecosystem is dependent upon it never changing. I cannot seem to find much discussion about this. I have found a no-GIL interpreter that works with numpy, scikit, etc. [2][3] so it doesn't seem to be a hard limit. (That said, it was not stated if that particular no-GIL implementation requires specially built versions of C-API libs or if it's a drop-in replacement.) [0]: https://docs.python.org/3/c-api/stable.html#c-api-stability https://docs.python.org/3/c-api/stable.html#c-api-stability [1]: https://docs.python.org/3/c-api/stable.html#platform-considerations https://docs.python.org/3/c-api/stable.html#platform-conside... [2]: https://github.com/colesbury/nogil https://github.com/colesbury/nogil [3]: https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/28 https://discuss.python.org/t/pep-703-making-the-global-inter...
- PhilipRoman 3y ago
  >multi-threading is the future Haha, reminds me of an image I saw with a farmer from some developing country saying "irrigation is the future". For everyone else multithreading has been the status quo for quite a long time.
- ActorNightly 3y ago
  >multi-threading is the future. Yes, with ML powered compilers recognizing what you are trying to do and generating the actual multithreaded code for you. And it won't be multithreaded code like you know it, in the sense of os specific threading code with context switching and what not. It will be compiled compute graphs targeted at specific ML hardware, likely with static addressing.
vb-8448 3y ago
> There is going to be a performance impact on single threaded code if the "no GIL" project is merged, something in the region of 10%. 10% doesn't look too much to me, I still don't get why today people care so much about single thread performance.
- samwillis 3y ago
  Exactly, and the thing is, the Faster Python project will completely surpass that 10% performance change.
  - _kulang 3y ago
    Sometimes people see it as losing 10% performance that you never get back
- awill 3y ago
  because a lot of code is single-threaded
- celeritascelery 3y ago
  Single threaded performance is still more important than multi-threaded. Most applications are single threaded, and single threaded programs are much easier to write and debug. Removing the GIL from python will not change that. If no-GIL has a 10% single thread performance hit, that means that essentially all my existing python code would be that much worse.
  - hospitalJail 3y ago
    Are your current python programs slow and it matters? Why havent you implemented multithreading? (don't get me wrong, I know the cost of implementation, but if speed matters, multithreading is a very reasonable step in python)
    - birdyrooster 3y ago
      Because the GIL and also you have to use tons of locks to get around the lack of thread safety for pythons objects.
meepmorp 3y ago
> There is also the argument that it could "divide the community" as some C extensions may not be ported to the new ABI that the no GIL project will result in. However again I'm unconvinced by that, the Python community has been through worse (Python 3) and even asyncIO completely divides the community now. I think the fact that you can name two other recent things which have divided the community is a solid argument for being a least a little gunshy about making big, breaking changes. There's the cost of the changes themselves, but there's also a cost to the language as a whole to add yet-another-upheaval. Performance is important, but not breaking things is also important. I can understand the appeal of doing something suboptimal (but better than current) in favor of not introducing a bunch of harder to predict side effects, both in code and the community.
- deschutes 3y ago
  Do sub interpreters actually work with c extensions? I get the extension API has long supported it. However, I wonder if in practice extensions rely on process global state to stash information. If so, sub interpreters invite all kinds of nasty bugs. Keep in mind that porting the most popular extensions is an easy exercise so the more interesting question is how this hidden majority of extensions fares.
p5a0u9l 3y ago
Removing the GIL should be seen as a last option.
matsemann 3y ago
GIL is one of the things that make Python an annoyance to work with. In saner languages, you could handle multiple requests at the same time, or easily spin something off in a thread to work in the background. In Python you can't do this. You need to duplicate your process, then pay the price of memory usage and other things multiple processes hinder (like communication between threads or pre-computed values now aren't shared so you need something external again). To deploy your app, you end up with 10 different deploys because each of them have to have a different entry point and separate task to fulfill.
- stinos 3y ago
  GIL is one of the things that make Python an annoyance to work with For your particular usecase, yes. Personally I've been using Python for like 20 years for various tasks and so far never got really bothered by the presence of it once. Worst case was having to wait somewhat longer for things to complete. For my case: still worth it compared to making things multithreaded. And async fixed the rest. And the things which I actually need to be fast aren't usually in Python anyway. I'm not saying the GIL should stay, it's just that it doesn't seem as much of a problem in the general land of Python. Or in other words: how many Python users out there even know what GIL means and does?
  - stult 3y ago
    > For your particular usecase, yes. The use case they are describing is a standard web server or web application. That's a pretty important and widely applicable use case to dismiss out of hand as "your particular usecase".
    - nomel 3y ago
      > The use case they are describing is a standard web server or web application. I believe this is what they were referring to when they said “async fixed the rest”.
    - nunuvit 3y ago
      The dismissiveness really goes the other way. Pythons like IronPython and Jython don't have a GIL. CPython does because it's primarily a glue language for extensions that might not be thread-safe. Web apps were given huge accommodation with async, so you can't say their needs are being dismissed. Why must we break the C in CPython for a use-case that could use one of the GIL-free Pythons?
RobotToaster 3y ago
Why not just try to make multiprocessing easier?
gazpacho 3y ago
The other thing I don't get is that the whole sub interpreters thing seems to totally break extension modules as well: https://github.com/PyO3/pyo3/issues/2274 https://github.com/PyO3/pyo3/issues/2274. In theory parts of sub-interpreters have been around for a while and it just happens that every extension module out there is incompatible with it because no one used it. But if it's going to become the recommended way to do parallelism going forward then they'll have to become compatible with it. The serialization thing is also a huge issue. Half of the time I want to use multiprocessing I end up finding that the serialization of data is the bottleneck and have to somehow re-architect my code to minimize it. I would much prefer a world in which asyncio is 2x faster and can benefit from real parallelism across threads. Libraries like anyio already make it super easy to work with async + threads. It would make Python a viable option for workloads where it currently just isn't.
- rogerbinns 3y ago
  (Disclosure: Author of a Python C extension that wraps SQLite that has 2 decades of development behind it.) Have a look at the current documentation for writing extensions. This approach is essentially unchanged since Python 2.0. https://docs.python.org/3/extending/newtypes_tutorial.html https://docs.python.org/3/extending/newtypes_tutorial.html In particular note how everything is declared static - ie only one instance of that data item will exist. If there are multiple interpreters then there needs to be one instance per sub-interpreter. That means no more static and initialisation has to be changed to attach these to the module object which is then attached to a specific interpreter instance. It also means every location you needed to access a previously static item (which often happens) has to change from a direct reference through new APIs chasing back from objects to get their owning module and then get the reference. That is the code churn the PyO3 issue is having to address. One bonus however is that you can then cleanly unload modules. This may still not be sufficient. For example I wrap SQLite and it has some global items like a logging callback. If my module was loaded into two different sub interpreters and both registered the logging callback, only one would win. These kind of gnarly issues are hard to discover and diagnose. Removing the GIL also won't magically help. I already release it at every possible opportunity. If it did go away, I would have to reintroduce a lock anyway to prevent concurrency in certain places. And there would have to be more locking around various Python data structures. For example if I am processing items in a list, I'd need a lock to prevent the list changing while processing. Currently the GIL handles that and ensures fewer bugs. I've also experienced the serialization overhead with multiprocessing. I made a client's code so much faster that any form of Python concurrency was slower because of all the overhead. I had to rearchitect the code to work on batches of data items instead of the far more natural one at a time. That finally allowed a performance improvement with multiprocessing.
dekhn 3y ago
The GIL will never be removed from the main python implementation. Histortically, the main value of GIL removal proposals and implementations has been to spur the core team to speed up single core codes. I think it's too late to consider removing the gil from the main implementation. Like guido said in the PEP thread, the python core team burned the community for 10 years with the 2-3 switch, and a GIL change would be likely as impactful; we'd have 10 years of people complaining their stuff didn't work. Frankly I wish Guido would just come out and tell Sam "no, we can't put this in cpython. You did a great work but compatibility issues trump performance". Kind of a shame because Hugunin implemented a Python on top of the CLR some 20 years ago and showed some extremely impressive performance results. Like jython, and pypy and other implementations, it never caught on because compatibility with cpython is one of the most important criteria for people dealing with lots of python code.
- giantrobot 3y ago
  > Like jython, and pypy and other implementations, it never caught on because compatibility with cpython is one of the most important criteria for people dealing with lots of python code. I think this is more an issue of popular packages were developed and tested against cpython and there wasn't enough effort available to port/test them against anything else. There's no special magic in cpython that Python programmers love, they just want their code to run. If they've got a numpy dependency (IIRC it doesn't support pypy but I'm not going to look it up so I may be corrected on that point this is a long parenthetical) they can't use an interpreter that doesn't support it. Even if it worked but had bugs it didn't have in cpython, they're still going to use cpython. Most people aren't writing Python for its super duper fast performance so they're fine leaving a little performance on the table by using the interpreter that their dependencies support. Whatever that is.
  - dehrmann 3y ago
    Jython doesn't have a good story for running native libraries like Numpy.
    - giantrobot 3y ago
zerkten 3y ago
This feels like it's playing out as I expected. I followed Python, and the Python community, really closely from 2008-2016 when there were tons of relatively small scale experiments happening. This all happened organically to a large extent and there was no one coordinating a grand vision. It seems like we have a continuation of this giving rise to the concern that there is some battle. I suspect there will be some butting of heads for a while before they work things out after seeing how the community reacts. All of this could be handled better with some thoughtful proactive engagement, but that's not really how things operate and there is no one to really enforce it.
jillesvangurp 3y ago
The GIL has been a blocker for many years. It's nice that the team is making progress of course. IMHO it's one of those bandaids they need to rip off. I was listening to the interview with Chris Lattner with Lex Friedman a week ago or so. Very interesting discussion on his project mojo which intends to build a new language that is backwards compatible and a drop in replacement for python with opt in strict typing, better support for native/primitive types where this makes sense, easier integrations with hardware optimizations, and of course no GIL. The idea would be that the migration path for existing code is that it should just work and then you optimize it and provide the compiler with type hints and other information so it can do a better job. Very ambitious roadmap and I'm curious to see if they'll be able to deliver. The main goal seems to be to enable programmers to do the things you currently can't do in python because it's too slow in python without running into a brick wall in terms of performance. I mostly work with JVM languages and a few other things but I occasionally do a bit with python as well. I've always liked it as a language but I'm by no means an expert in it. I recently spent a day building a simple geocoder and since I know about the GIL, I went straight for the multi processing library and did not bother with threads. IMHO there's absolutely no point in attempting to use threads with python with the GIL in place. I needed to geocode a few hundred thousand things in a reasonable time frame, so all I wanted to do was use a few different processes concurrently so I could cut down the runtime to something reasonable. Python is ok for single threaded stuff but you run into a brickwall doing anything with multiple processes or threads and juggling state. In the end I just gave up and wrote a bunch of logic that splits the input into files, processes the files with separate processes, waits for that to finish, and then combines the output files. Just a lot of silly boiler plate and abusing the file system for sharing state. It does what it needs to but it feels a bit primitive and backwards and I'm not proud of the solution. Removing the GIL, adding some structured concurrency, and maybe some other features, would make python a lot more capable for data processing. And since there are a lot of people already use python for that sort of thing, I don't think that would be such a bad thing. Data science and data processing are the core use case for python. I don't think people actually care a lot about the raw python performance. It's never been that great to begin with. If it's performance critical, it's mostly being done via native libraries already.
- robertlagrant 3y ago
gpderetta 3y ago
If the GIL were an optional interpreter parameter you could spawn GILed subinterpreters and GILless subinterpreters according to your needs.
dist-epoch 3y ago
As someone who observed Python core development for many years, a major change to the interpreter REQUIRES core-dev buy in. There have been at least 5 big projects which proposed large changes, they have all been declined. It is a NIH syndrome, if a big project doesn't originate in the dev team, it will not be accepted.
btilly 3y ago
My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem. Also, no matter how much you wish it otherwise, retrofitting concurrency on an existing project guarantees that you'll wind up with subtle concurrency bugs. You might not encounter them often, and they're hard to spot, but they'll be there. Furthermore existing libraries that expect to be single-threaded are now a potential source of concurrency bugs. And there is no particular reason to expect the authors of said libraries to have either the skills or interest to track those bugs down and fix them. Nor do I expect that multi-threaded enthusiasts who use those libraries in unwise ways will recognize the potential problems. Particularly not in a dynamic language like Python that doesn't have great tool support for tracking down such bugs in an automated way. As a result if "no GIL" ever gets merged, I expect that the whole Python ecosystem will get much worse as well. But that's no skin off of my back - I've learned plenty of languages. I can simply learn one that hasn't (yet) made this category of mistake.
- coldtea 3y ago
  >My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem. It's 2023 already. 1988 called.
  - btilly 3y ago
    Did you know that Coverity actively REMOVED checks for concurrency bugs? It turns out that when the programmer doesn't understand what the tool says, managers believe the programmer and throw out the tool. Coverity was finding itself in situations where they were finding real bugs, and being punished for it by losing the sale. So they removed the checks for those bugs. I'll revisit my opinion of multithreaded code when things like that stop happening. In the meantime there are models of how to run code on multiple threads that work well enough with different primitives. See Erlang, Go, and Rust for three of them. Also, if you squint sideways, microservices. (Though most people set up microservices in a way that makes debugging problematic. Topic for another day.)
coldtea 3y ago
>There is also the argument that it could "divide the community" as some C extensions may not be ported to the new ABI that the no GIL project will result in I think the arguments are a red herring. It's more rationalizations for not wanting to do it.
samsquire 3y ago
I have often wondered what the solution to the serialisation of objects between subinterpreters is. If its garbage collection that's the problem, I think you could transfer ownership between threads, so the subinterpreter takes ownership of the object and all references to it in the source interpreter are voided. Alternatively you can do something like Java and all objects are in a global object allocator, passing things between threads doesn't require interpretation, just a reference.
ptx 3y ago
If we could just get efficient passing of objects graphs from one subinterpreter to another, which is not in the current plan, I think that would solve a lot of use cases. That would allow producer/consumer-style processing with multiple cores without the serialization overhead of the multiprocessing module. Removing the GIL seems like it could make things more complicated in many ways by making lots of currently thread-safe code subtly unsafe, but I might be wrong about this. (...in which case it would just make things very slow because everything is synchronized?)
bastawhiz 3y ago
> Their argument is that the "sub interpreters" they are adding (each with its own GIL) will fulfil the same use cases of multithreaded code without a GIL, but they still have the overhead of encoding and passing data in the same way you have to with sub processors. This is smart, though, because (even if it's not great) there's a lot of evidence that it works in practice. Specifically, this is almost exactly what JavaScript does with workers. It's not a great API and it's cumbersome to write code for, but it got implemented successfully and people use it successfully (and it didn't slow down the whole web).
samwillis 3y ago
This just posted on the Python forum is a brilliant rundown of the conflicting "Faster Python" and "No GIL" projects, and a proposal (plus call for funding) for a route forward. I think everyone would agree that trying to combine both would be ideal! "A fast, free threading Python" https://discuss.python.org/t/a-fast-free-threading-python/27903 https://discuss.python.org/t/a-fast-free-threading-python/27...
- jrochkind1 3y ago
  this is a good read and deserves to be on the homepage with it's own thread too!
- dragonwriter 3y ago
  It is from the person most involved in the faster-with-GIL effort, and its recommendation is to prioritize that effort in any case, and if the resources are available for that and no-gil, do both. Not that I disagree with the recommendation, but one of the sides saying “as long as resources make us choose, choose my side” is...not really surprising.
  - samwillis 3y ago
    Not surprising, but I'm very happy they are trying to find a route forward for both. That I commend. I think from memory "Faster Python" is Microsoft funded and "No GIL" is funded by Facebook. If they can find a way to fund a combined effort that would be good. I suspect the conflicting funding also adds to the general political difficulty around this.
nmstoker 3y ago
I do hope the dialogue stays cordial, constructive and open rather than becoming distinct entrenched camps - the Python community has a strong and mature community spirit so this seems plausible and not too much wishful thinking. Much as No GIL would be an adventure, I'm leaning towards the more gradual and stable changes from the FasterPython team and I can see that throwing No GIL into the mix adds complexity at an inopportune moment.
AlphaSite 3y ago
Nogil would give far larger returns and I wish they’d focus on that. That’s the best way to a faster python.
JohnFen 3y ago
Oh, boy. Will any of that impact backward compatibility? I don't develop anything in Python, but it is used by several applications of importance to me. The lack of compatibility between versions is a thing that bites me hard, and I tend to curse Python because of it.
hklzr100 3y ago
Where is the progress? They claim "10-60%" improvements over Python 3.9 (I believe) and I don't notice much, partly because Python has always been sped up by C-extensions. The price is an added complexity of the code base. I truly don't understand why Python always gets a pass and people applaud every announcement, no matter how trivial or elusive it is. The GIL effort is another matter. I'd rather have a simple interpreter with no-GIL than this mess of relatively small speedups. But like other GvR efforts where he presided over a small group of people who did the work his efforts will of course go in. Like asyncio, the suboptimal "match" statement, the peg parser (which adds new workloads for other implementations because the de-facto DFL cannot be bothered with publishing an LALR grammar). Python is a glue language, people can go elsewhere for speed.
geekraver 3y ago
It’s overstating the case to call this a “struggle” between factions; it’s an important discussion with a lot of ramifications (and while it is unresolved a lot of work is stalled). https://discuss.python.org/t/a-fast-free-threading-python/27903 https://discuss.python.org/t/a-fast-free-threading-python/27...

bratao 3y ago

Yeah, they clearly stated that here https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/118 https://discuss.python.org/t/pep-703-making-the-global-inter... I really wish that GIL go away. It is better to pay this price now, multi-threading is the future.

hospitalJail 3y ago
I'm so conflicted. ~40 of the programs I am responsible for are single threaded. They were relatively quick to develop and were made by Electrical Engineers rather than career/degreed programmers. 2 programs use multithreading, I had to do that. The learning curve was not a huge deal, but the development time adds at least hours. In my case days(due to testing). I imagine its too hard to have an optional flag at the start of each program that can let the user decide?
- birdyrooster 3y ago
  The problem is that Python types are not thread safe so you have to jump through more hoops to have safe parallelization in Python. These changes would make it so that writing multithreaded code will be much easier it seems like.
- maleldil 3y ago
  > I imagine its too hard to have an optional flag at the start of each program that can let the user decide? Adding nogil would mean deep changes to the interpreter. I imagine maintaining both versions would be almost like forking the project.
- coldtea 3y ago
  >but the development time adds at least hours So? some "hours of development time" is nothing.
  - sfink 3y ago
    True, and as you get better at it, those hours will be fewer. The weeks of debugging never go away, though. (Or at least, not as long as you're using shared state, but that's really the only thing under consideration here.)
  - hospitalJail 3y ago
    I'm mostly with you. I think this probably affects part time/newbie programmers more. A few hours of dev time is 1 or 2 nights of work.
- dragonwriter 3y ago
  > I imagine its too hard to have an optional flag at the start of each program that can let the user decide? The actual next-step proposal toward no-gil is GIL as a build-time flag (which isn’t quite the same as a runtime flag, but not too far off, either.) https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
- nunuvit 3y ago
  It's not like CPython is depriving anyone of GIL-free Python. IronPython and Jython are GIL-free. You can have it yesterday. CPython maintains Python's original purpose as a glue language. You know how much worse CPython would be for that use-case without the GIL. GVR has said he'll support removing the GIL as long as there's no performance hit. Otherwise, it's simply asking for a fundamental change of direction and too much sacrifice from everyone who needs CPython specifically, just to improve the productivity of people who could use a different Python.
sgt 3y ago
Not for Python, I feel. In sheer volume, the vast majority of my Python programs are single-threaded. I want my programs to be very quick in runtime when they run. Those that are multi-threaded are seeing minor to medium load. If expecting extreme load (like Twitter scale), then Python is usually not the answer (rather go to a statically typed language like Java, Go, Rust etc).
- gpderetta 3y ago
  > In sheer volume, the vast majority of my Python programs are single-threaded obviously if multithreading is near-useless in python, very few programs will take advantage of it.
  - zo1 3y ago
    Likewise I can say: Obviously we still have the GIL because very few people want it removed.
    - gpderetta 3y ago
      Well, yes, that's pretty much obvious: the majority of python programmers do not care, those that cared have moved on as it is easier than changing the language. And I'll be first to admit that improving single threaded performance has an higher priority than removing the GIL.
- AlphaSite 3y ago
  Is single threaded perf is important, you’ve already lost by using python. You’re only ever going to get, ok-ish performance or slightly more ok-ish
- gcbirzan 3y ago
  > Not for Python, I feel. In sheer volume, the vast majority of my Python programs are single-threaded. Yes, they are single threaded, because using multiple threads brings very little benefit in most cases... > If expecting extreme load (like Twitter scale), then Python is usually not the answer (rather go to a statically typed language like Java, Go, Rust etc). So that means we shouldn't get any performance improvements, because there are faster languages out there?
amelius 3y ago
It's relatively simple to make the GIL go away: just compile to some VM that has a good concurrent garbage collector would be one approach. Yes, this will break some assumptions here and there, but not too difficult to overcome especially if you bump the version number to Python 4. However, that leaves a lot of C code that you can't talk to anymore because the C code requires the old Python FFI. I think this is where the main problem lies.
- notatallshaw 3y ago
  > It's relatively simple to make the GIL go away: just compile to some VM that has a good concurrent garbage collector would be one approach Sure, if you don't mind paying a 50-90% performance impact on single threaded performance or completely abandon C-API compatibility and have C extensions start from scratch then there are simple approaches. If you look at any example in the past to remove the GIL you would see that keeping these two requirements of not having terrible single threaded performance and not having almost a completely new C-API is actually very complex and takes a lot of expertise to implement.
  - saltminer 3y ago
    This might be a dumb question, but why would removing the GIL break FFI? Is it just that existing no-GIL implementations/proposals have discarded/ignored it, or is there a fundamental requirement, e.g. C programs unavoidably interact directly with the GIL? (In which case, couldn't a "legacy FFI" wrapper be created?) I know that the C-API is only stable between minor releases [0] compiled in the same manner [1], so it's not like the ecosystem is dependent upon it never changing. I cannot seem to find much discussion about this. I have found a no-GIL interpreter that works with numpy, scikit, etc. [2][3] so it doesn't seem to be a hard limit. (That said, it was not stated if that particular no-GIL implementation requires specially built versions of C-API libs or if it's a drop-in replacement.) [0]: https://docs.python.org/3/c-api/stable.html#c-api-stability https://docs.python.org/3/c-api/stable.html#c-api-stability [1]: https://docs.python.org/3/c-api/stable.html#platform-considerations https://docs.python.org/3/c-api/stable.html#platform-conside... [2]: https://github.com/colesbury/nogil https://github.com/colesbury/nogil [3]: https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/28 https://discuss.python.org/t/pep-703-making-the-global-inter...
PhilipRoman 3y ago
>multi-threading is the future Haha, reminds me of an image I saw with a farmer from some developing country saying "irrigation is the future". For everyone else multithreading has been the status quo for quite a long time.
ActorNightly 3y ago
>multi-threading is the future. Yes, with ML powered compilers recognizing what you are trying to do and generating the actual multithreaded code for you. And it won't be multithreaded code like you know it, in the sense of os specific threading code with context switching and what not. It will be compiled compute graphs targeted at specific ML hardware, likely with static addressing.

vb-8448 3y ago

> There is going to be a performance impact on single threaded code if the "no GIL" project is merged, something in the region of 10%. 10% doesn't look too much to me, I still don't get why today people care so much about single thread performance.

samwillis 3y ago
Exactly, and the thing is, the Faster Python project will completely surpass that 10% performance change.
- _kulang 3y ago
  Sometimes people see it as losing 10% performance that you never get back
awill 3y ago
because a lot of code is single-threaded
celeritascelery 3y ago
Single threaded performance is still more important than multi-threaded. Most applications are single threaded, and single threaded programs are much easier to write and debug. Removing the GIL from python will not change that. If no-GIL has a 10% single thread performance hit, that means that essentially all my existing python code would be that much worse.
- hospitalJail 3y ago
  Are your current python programs slow and it matters? Why havent you implemented multithreading? (don't get me wrong, I know the cost of implementation, but if speed matters, multithreading is a very reasonable step in python)
  - birdyrooster 3y ago
    Because the GIL and also you have to use tons of locks to get around the lack of thread safety for pythons objects.

meepmorp 3y ago

> There is also the argument that it could "divide the community" as some C extensions may not be ported to the new ABI that the no GIL project will result in. However again I'm unconvinced by that, the Python community has been through worse (Python 3) and even asyncIO completely divides the community now. I think the fact that you can name two other recent things which have divided the community is a solid argument for being a least a little gunshy about making big, breaking changes. There's the cost of the changes themselves, but there's also a cost to the language as a whole to add yet-another-upheaval. Performance is important, but not breaking things is also important. I can understand the appeal of doing something suboptimal (but better than current) in favor of not introducing a bunch of harder to predict side effects, both in code and the community.

deschutes 3y ago
Do sub interpreters actually work with c extensions? I get the extension API has long supported it. However, I wonder if in practice extensions rely on process global state to stash information. If so, sub interpreters invite all kinds of nasty bugs. Keep in mind that porting the most popular extensions is an easy exercise so the more interesting question is how this hidden majority of extensions fares.

p5a0u9l 3y ago

Removing the GIL should be seen as a last option.

matsemann 3y ago

GIL is one of the things that make Python an annoyance to work with. In saner languages, you could handle multiple requests at the same time, or easily spin something off in a thread to work in the background. In Python you can't do this. You need to duplicate your process, then pay the price of memory usage and other things multiple processes hinder (like communication between threads or pre-computed values now aren't shared so you need something external again). To deploy your app, you end up with 10 different deploys because each of them have to have a different entry point and separate task to fulfill.

stinos 3y ago
GIL is one of the things that make Python an annoyance to work with For your particular usecase, yes. Personally I've been using Python for like 20 years for various tasks and so far never got really bothered by the presence of it once. Worst case was having to wait somewhat longer for things to complete. For my case: still worth it compared to making things multithreaded. And async fixed the rest. And the things which I actually need to be fast aren't usually in Python anyway. I'm not saying the GIL should stay, it's just that it doesn't seem as much of a problem in the general land of Python. Or in other words: how many Python users out there even know what GIL means and does?
- stult 3y ago
  > For your particular usecase, yes. The use case they are describing is a standard web server or web application. That's a pretty important and widely applicable use case to dismiss out of hand as "your particular usecase".
  - nomel 3y ago
    > The use case they are describing is a standard web server or web application. I believe this is what they were referring to when they said “async fixed the rest”.
  - nunuvit 3y ago
    The dismissiveness really goes the other way. Pythons like IronPython and Jython don't have a GIL. CPython does because it's primarily a glue language for extensions that might not be thread-safe. Web apps were given huge accommodation with async, so you can't say their needs are being dismissed. Why must we break the C in CPython for a use-case that could use one of the GIL-free Pythons?

RobotToaster 3y ago

Why not just try to make multiprocessing easier?

gazpacho 3y ago

The other thing I don't get is that the whole sub interpreters thing seems to totally break extension modules as well: https://github.com/PyO3/pyo3/issues/2274 https://github.com/PyO3/pyo3/issues/2274. In theory parts of sub-interpreters have been around for a while and it just happens that every extension module out there is incompatible with it because no one used it. But if it's going to become the recommended way to do parallelism going forward then they'll have to become compatible with it. The serialization thing is also a huge issue. Half of the time I want to use multiprocessing I end up finding that the serialization of data is the bottleneck and have to somehow re-architect my code to minimize it. I would much prefer a world in which asyncio is 2x faster and can benefit from real parallelism across threads. Libraries like anyio already make it super easy to work with async + threads. It would make Python a viable option for workloads where it currently just isn't.

rogerbinns 3y ago
(Disclosure: Author of a Python C extension that wraps SQLite that has 2 decades of development behind it.) Have a look at the current documentation for writing extensions. This approach is essentially unchanged since Python 2.0. https://docs.python.org/3/extending/newtypes_tutorial.html https://docs.python.org/3/extending/newtypes_tutorial.html In particular note how everything is declared static - ie only one instance of that data item will exist. If there are multiple interpreters then there needs to be one instance per sub-interpreter. That means no more static and initialisation has to be changed to attach these to the module object which is then attached to a specific interpreter instance. It also means every location you needed to access a previously static item (which often happens) has to change from a direct reference through new APIs chasing back from objects to get their owning module and then get the reference. That is the code churn the PyO3 issue is having to address. One bonus however is that you can then cleanly unload modules. This may still not be sufficient. For example I wrap SQLite and it has some global items like a logging callback. If my module was loaded into two different sub interpreters and both registered the logging callback, only one would win. These kind of gnarly issues are hard to discover and diagnose. Removing the GIL also won't magically help. I already release it at every possible opportunity. If it did go away, I would have to reintroduce a lock anyway to prevent concurrency in certain places. And there would have to be more locking around various Python data structures. For example if I am processing items in a list, I'd need a lock to prevent the list changing while processing. Currently the GIL handles that and ensures fewer bugs. I've also experienced the serialization overhead with multiprocessing. I made a client's code so much faster that any form of Python concurrency was slower because of all the overhead. I had to rearchitect the code to work on batches of data items instead of the far more natural one at a time. That finally allowed a performance improvement with multiprocessing.

dekhn 3y ago

The GIL will never be removed from the main python implementation. Histortically, the main value of GIL removal proposals and implementations has been to spur the core team to speed up single core codes. I think it's too late to consider removing the gil from the main implementation. Like guido said in the PEP thread, the python core team burned the community for 10 years with the 2-3 switch, and a GIL change would be likely as impactful; we'd have 10 years of people complaining their stuff didn't work. Frankly I wish Guido would just come out and tell Sam "no, we can't put this in cpython. You did a great work but compatibility issues trump performance". Kind of a shame because Hugunin implemented a Python on top of the CLR some 20 years ago and showed some extremely impressive performance results. Like jython, and pypy and other implementations, it never caught on because compatibility with cpython is one of the most important criteria for people dealing with lots of python code.

giantrobot 3y ago
> Like jython, and pypy and other implementations, it never caught on because compatibility with cpython is one of the most important criteria for people dealing with lots of python code. I think this is more an issue of popular packages were developed and tested against cpython and there wasn't enough effort available to port/test them against anything else. There's no special magic in cpython that Python programmers love, they just want their code to run. If they've got a numpy dependency (IIRC it doesn't support pypy but I'm not going to look it up so I may be corrected on that point this is a long parenthetical) they can't use an interpreter that doesn't support it. Even if it worked but had bugs it didn't have in cpython, they're still going to use cpython. Most people aren't writing Python for its super duper fast performance so they're fine leaving a little performance on the table by using the interpreter that their dependencies support. Whatever that is.
- dehrmann 3y ago
  Jython doesn't have a good story for running native libraries like Numpy.
  - giantrobot 3y ago

zerkten 3y ago

This feels like it's playing out as I expected. I followed Python, and the Python community, really closely from 2008-2016 when there were tons of relatively small scale experiments happening. This all happened organically to a large extent and there was no one coordinating a grand vision. It seems like we have a continuation of this giving rise to the concern that there is some battle. I suspect there will be some butting of heads for a while before they work things out after seeing how the community reacts. All of this could be handled better with some thoughtful proactive engagement, but that's not really how things operate and there is no one to really enforce it.

jillesvangurp 3y ago

The GIL has been a blocker for many years. It's nice that the team is making progress of course. IMHO it's one of those bandaids they need to rip off. I was listening to the interview with Chris Lattner with Lex Friedman a week ago or so. Very interesting discussion on his project mojo which intends to build a new language that is backwards compatible and a drop in replacement for python with opt in strict typing, better support for native/primitive types where this makes sense, easier integrations with hardware optimizations, and of course no GIL. The idea would be that the migration path for existing code is that it should just work and then you optimize it and provide the compiler with type hints and other information so it can do a better job. Very ambitious roadmap and I'm curious to see if they'll be able to deliver. The main goal seems to be to enable programmers to do the things you currently can't do in python because it's too slow in python without running into a brick wall in terms of performance. I mostly work with JVM languages and a few other things but I occasionally do a bit with python as well. I've always liked it as a language but I'm by no means an expert in it. I recently spent a day building a simple geocoder and since I know about the GIL, I went straight for the multi processing library and did not bother with threads. IMHO there's absolutely no point in attempting to use threads with python with the GIL in place. I needed to geocode a few hundred thousand things in a reasonable time frame, so all I wanted to do was use a few different processes concurrently so I could cut down the runtime to something reasonable. Python is ok for single threaded stuff but you run into a brickwall doing anything with multiple processes or threads and juggling state. In the end I just gave up and wrote a bunch of logic that splits the input into files, processes the files with separate processes, waits for that to finish, and then combines the output files. Just a lot of silly boiler plate and abusing the file system for sharing state. It does what it needs to but it feels a bit primitive and backwards and I'm not proud of the solution. Removing the GIL, adding some structured concurrency, and maybe some other features, would make python a lot more capable for data processing. And since there are a lot of people already use python for that sort of thing, I don't think that would be such a bad thing. Data science and data processing are the core use case for python. I don't think people actually care a lot about the raw python performance. It's never been that great to begin with. If it's performance critical, it's mostly being done via native libraries already.

robertlagrant 3y ago

gpderetta 3y ago

If the GIL were an optional interpreter parameter you could spawn GILed subinterpreters and GILless subinterpreters according to your needs.

dist-epoch 3y ago

As someone who observed Python core development for many years, a major change to the interpreter REQUIRES core-dev buy in. There have been at least 5 big projects which proposed large changes, they have all been declined. It is a NIH syndrome, if a big project doesn't originate in the dev team, it will not be accepted.

btilly 3y ago

My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem. Also, no matter how much you wish it otherwise, retrofitting concurrency on an existing project guarantees that you'll wind up with subtle concurrency bugs. You might not encounter them often, and they're hard to spot, but they'll be there. Furthermore existing libraries that expect to be single-threaded are now a potential source of concurrency bugs. And there is no particular reason to expect the authors of said libraries to have either the skills or interest to track those bugs down and fix them. Nor do I expect that multi-threaded enthusiasts who use those libraries in unwise ways will recognize the potential problems. Particularly not in a dynamic language like Python that doesn't have great tool support for tracking down such bugs in an automated way. As a result if "no GIL" ever gets merged, I expect that the whole Python ecosystem will get much worse as well. But that's no skin off of my back - I've learned plenty of languages. I can simply learn one that hasn't (yet) made this category of mistake.

coldtea 3y ago
>My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem. It's 2023 already. 1988 called.
- btilly 3y ago
  Did you know that Coverity actively REMOVED checks for concurrency bugs? It turns out that when the programmer doesn't understand what the tool says, managers believe the programmer and throw out the tool. Coverity was finding itself in situations where they were finding real bugs, and being punished for it by losing the sale. So they removed the checks for those bugs. I'll revisit my opinion of multithreaded code when things like that stop happening. In the meantime there are models of how to run code on multiple threads that work well enough with different primitives. See Erlang, Go, and Rust for three of them. Also, if you squint sideways, microservices. (Though most people set up microservices in a way that makes debugging problematic. Topic for another day.)

coldtea 3y ago

>There is also the argument that it could "divide the community" as some C extensions may not be ported to the new ABI that the no GIL project will result in I think the arguments are a red herring. It's more rationalizations for not wanting to do it.

samsquire 3y ago

I have often wondered what the solution to the serialisation of objects between subinterpreters is. If its garbage collection that's the problem, I think you could transfer ownership between threads, so the subinterpreter takes ownership of the object and all references to it in the source interpreter are voided. Alternatively you can do something like Java and all objects are in a global object allocator, passing things between threads doesn't require interpretation, just a reference.

ptx 3y ago

If we could just get efficient passing of objects graphs from one subinterpreter to another, which is not in the current plan, I think that would solve a lot of use cases. That would allow producer/consumer-style processing with multiple cores without the serialization overhead of the multiprocessing module. Removing the GIL seems like it could make things more complicated in many ways by making lots of currently thread-safe code subtly unsafe, but I might be wrong about this. (...in which case it would just make things very slow because everything is synchronized?)

bastawhiz 3y ago

> Their argument is that the "sub interpreters" they are adding (each with its own GIL) will fulfil the same use cases of multithreaded code without a GIL, but they still have the overhead of encoding and passing data in the same way you have to with sub processors. This is smart, though, because (even if it's not great) there's a lot of evidence that it works in practice. Specifically, this is almost exactly what JavaScript does with workers. It's not a great API and it's cumbersome to write code for, but it got implemented successfully and people use it successfully (and it didn't slow down the whole web).

samwillis 3y ago

This just posted on the Python forum is a brilliant rundown of the conflicting "Faster Python" and "No GIL" projects, and a proposal (plus call for funding) for a route forward. I think everyone would agree that trying to combine both would be ideal! "A fast, free threading Python" https://discuss.python.org/t/a-fast-free-threading-python/27903 https://discuss.python.org/t/a-fast-free-threading-python/27...

jrochkind1 3y ago
this is a good read and deserves to be on the homepage with it's own thread too!
dragonwriter 3y ago
It is from the person most involved in the faster-with-GIL effort, and its recommendation is to prioritize that effort in any case, and if the resources are available for that and no-gil, do both. Not that I disagree with the recommendation, but one of the sides saying “as long as resources make us choose, choose my side” is...not really surprising.
- samwillis 3y ago
  Not surprising, but I'm very happy they are trying to find a route forward for both. That I commend. I think from memory "Faster Python" is Microsoft funded and "No GIL" is funded by Facebook. If they can find a way to fund a combined effort that would be good. I suspect the conflicting funding also adds to the general political difficulty around this.

nmstoker 3y ago

I do hope the dialogue stays cordial, constructive and open rather than becoming distinct entrenched camps - the Python community has a strong and mature community spirit so this seems plausible and not too much wishful thinking. Much as No GIL would be an adventure, I'm leaning towards the more gradual and stable changes from the FasterPython team and I can see that throwing No GIL into the mix adds complexity at an inopportune moment.

AlphaSite 3y ago

Nogil would give far larger returns and I wish they’d focus on that. That’s the best way to a faster python.

JohnFen 3y ago

Oh, boy. Will any of that impact backward compatibility? I don't develop anything in Python, but it is used by several applications of importance to me. The lack of compatibility between versions is a thing that bites me hard, and I tend to curse Python because of it.

hklzr100 3y ago

Where is the progress? They claim "10-60%" improvements over Python 3.9 (I believe) and I don't notice much, partly because Python has always been sped up by C-extensions. The price is an added complexity of the code base. I truly don't understand why Python always gets a pass and people applaud every announcement, no matter how trivial or elusive it is. The GIL effort is another matter. I'd rather have a simple interpreter with no-GIL than this mess of relatively small speedups. But like other GvR efforts where he presided over a small group of people who did the work his efforts will of course go in. Like asyncio, the suboptimal "match" statement, the peg parser (which adds new workloads for other implementations because the de-facto DFL cannot be bothered with publishing an LALR grammar). Python is a glue language, people can go elsewhere for speed.

geekraver 3y ago

It’s overstating the case to call this a “struggle” between factions; it’s an important discussion with a lot of ramifications (and while it is unresolved a lot of work is stalled). https://discuss.python.org/t/a-fast-free-threading-python/27903 https://discuss.python.org/t/a-fast-free-threading-python/27...