10 ms·

I always openly wonder with this proposal — how are they going to do this while making sure programs are still correct? So much existing multithreaded Python co

I always openly wonder with this proposal — how are they going to do this while making sure programs are still correct? So much existing multithreaded Python code is written in an unsafe manner. Specifically, talking about data races I’ve seen time and again in codebases across companies and OSS projects. The programs don’t break only because they implicitly rely on the GIL providing execution to a single thread at a time. If the GIL is gone, then these programs will break. And since Python is such a dynamically typed language, I seriously doubt that there exists a static analyzer that could identify these issues in existing Python programs. More likely, they’ll be insidious bugs that crop up at runtime in a non-deterministic fashion. Ideally leading to a crash, With this class of bugs, it’s likely to just result in incorrect operations being performed. Perhaps this GIL-less proposal isn’t actually intended to be used on the overwhelming majority of programs? Maybe it’s just a hyper specialized tool for a very few number of circumstances where the programmer knows, there won’t be a GIL, and can program against that fact?

plonk 3y ago
I think the point was to let libraries declare whether they support nogil mode (opt-in), and your program would only run with no GIL if all the dependencies allow it? So they have all the time in the world to iron out those bugs.
- eptcyka 3y ago
  At what point can an interpreter establish that a given python script will not be importing any more modules?
  - Borealid 3y ago
    If you import another module the gil could be re-enabled. Not saying that's what they will do, just what they could.
    - KMag 3y ago
      Right. It would be possible to implement the GIL as a readers-writer lock, where thread state includes a counter for the number of frames in the call stack that are within libraries not marked nogil. (Let's call these non-nogil libraries "GIL-dependent".) When the count goes from zero to one, the thread attempts to upgrade its reader lock to a writer lock. When its count goes from one to zero, the thread downgrades its lock from writer to reader. That way, there's at most one thread executing within GIL-dependent code at a time. Furthermore, if there is a thread executing within GIL-dependent code, all of the other threads are blocked waiting to acquire the GIL (in reader mode if they're nogil-safe, and writer mode if they're GIL-dependent.) As now, any thread holding the GIL in writer mode would need to drop the GIL when attempting to acquire any other lock (and re-acquire immediately afterward). To prevent starvation, one would presumably need a mechanism similar to periodic GC safepoints where nogil-safe threads still check if any thread is waiting to acquire the GIL in writer mode.
  - lacker 3y ago
    Perhaps it could just fail at runtime if you ever import a module that doesn't support nogil mode? AFAICT that's how it works if, for example, you run Python code that uses f-strings in a Python version that doesn't support f-strings.
    - tyingq 3y ago
      Python's support for run-time version/flag/setting detection isn't great for this kind of thing either. With Perl, for example, you get a BEGIN {} block that is run before it tries to parse the script...so you can detect and shim, etc. Python bombs before you can gain any control, because it parses the whole file. So you can separate things into modules to get around that, but it's not great when you want a simple one-file script.
      - siddheshgunjal 3y ago
        We experienced programmers who know how to separate things into modules and use it wisely at the correct place. But most of the beginners and intermediate developers always tend to do it in a "Everything in single script" manner which might also make it difficult to have more control over the application's behaviour.
        tyingq 3y ago
        You're making a lot of assumptions in a smug sort of way. There's plenty of spots where a single script makes sense, and plenty where it doesn't.
  - Too 3y ago
    If that is the only way. They need to change some semantics around import statements to not be runtime conditional. (Mypy can do it without running the code so to some extent it is possible). With that in place, each module can at the top declare #nogilsafe and the interpreter can know that no more modules will be loaded runtime. Dynamic imports via importlib will need other consideration. Expecting every transitive module to add this marker is very optimistic though. It’s thousands of packages with hundreds of modules each, that need to add this everywhere. Other languages have done similar journeys, like typescript “strict” added at top of each file. Except those are a lot more local, by not expecting all dependencies to follow.
  - plonk 3y ago
    I kind of thought Python would enable the GIL if any no-nogil libraries were installed, but even that is hard to define when you can modify sys.path with environment variables and code. Will look it up.
- tgv 3y ago
  Yes, but the plan is to remove the opt-in in time. That will put a lot of pressure on the eco system. I expect many libraries written in C or relying on C-based extensions to simply get dropped. Which will make that users will stay on the last GIL-supporting version. It's Python 2->3, but potentially worse.
  - plonk 3y ago
    In my important dependencies, there are deep learning frameworks from billion-dollar companies, bindings for C++ libraries that are basically standard in the field, projects from CS labs with millions of users. I don't see any of them getting abandoned. But I guess I can't judge how much work removing the GIL could take. The big projects tend to be well-written and well-maintained, for what it's worth. Which projects do you have in mind that have a significant user base, are still maintained, and would be too costly to port for someone to do the effort?
    - tgv 3y ago
      The small ones, of which there are many more. Written for some specific purpose, half maintained, used in 1 or 100 projects. Those will suffer. Or perhaps worse, they get fixed, but wrongly, and introduce subtle bugs when your application is under a somewhat heavier load. Good luck finding the offenders. You might not even see a bug, only bad or irreproducible results. Multi-threading, concurrency, and parallelism are fraught with problems. Your precious ML/DL libraries may not even be upgraded, because writing neural network code is not the same as writing thread-safe code. If it comes from a CS lab, its authors have already left, and there's nothing worthy of a publication in adding thread-safety. Certainly not when you can simply stick to Python 3.last-gil-version.
      - plonk 3y ago
        > Your precious ML/DL libraries may not even be upgraded, because writing neural network code is not the same as writing thread-safe code. If it comes from a CS lab, its authors have already left, and there's nothing worthy of a publication in adding thread-safety. PyTorch and sklearn won't stop being maintained though. I don't rely on unmaintained research code in production, I adapt what I need under MIT license. Any other way sounds crazy. Plus, most research code is very high-level and uses the same facilities (from e.g. PyTorch again) that everyone else uses, the actual distributed and multithreaded work happens in the main libraries. You'll still be able to use the same neural network code that worked before. I don't see a huge problem for people who already had their dependency list under control. If you had anything that's both hard to replace and not big enough to be upgraded though, I'd argue that it was always going to bite you at some point.
bratao 3y ago
The GIL only protect the Interpreter. The only thing it may do is to make it infrequently. There are MANY threading bugs in actual Python code.
deleted 3y ago
[deleted]
KeplerBoy 3y ago
Just a little fun fact: The GIL absolutely does not prevent all race condition bugs. Threads contending for the GIL can already steal it from each other at unfavorable times and cause havoc.
- ynik 3y ago
  In fact, it prevents very few race condition bugs. Even inside a C extension where the Python API feels like it gives you control over when you release the GIL (with functions you'd have to call explicitly to release the GIL), it turns out that: * any operation that allocates new Python objects might trigger garbage collection * garbage collection may run `__del__` of objects completely unrelated to the currently running C code * `__del__` can be implemented in python, thus releasing the GIL between bytecode instructions Thus there's a lot of (rarely exercised) potential for concurrency even in C extensions that don't explicitly release the GIL themselves. nogil will make it easier to trigger data race bugs, but many of them will already have been theoretically possible before.
amelius 3y ago
On the other hand, perhaps translating existing modules to a No-GIL API is tedious but straightforward, and something that can be done using automated tools (perhaps even LLMs).
baggiponte 3y ago
I suppose no-GIL Python will be here in no less than 3/4 release cycles. 3.11 has been out for a year and most Python code in prod is what, 3.8? So I guess we won’t have to deal with this at scale before idk 2030 is approaching. I also don’t see Python runtimes in prod being updated from whatever they’re on now to newest releases. I don’t want to sound harsh, but the SC stated they don’t want to have another 2-to-3 migration, so people won’t update lightly. Yes, most of the content online right now might be dangerous to copy paste
- sgarland 3y ago
  Tangentially, 3.11 was the first release in quite some time to have major speed improvements across the board. The average is 25%, sometimes far more. Anyone who hasn’t upgraded to it by now is needlessly spending extra on compute.
  - sapiogram 3y ago
    It's only 25% speedup in actual Python code, in domains like data science this won't matter much, because your cpu cycles are mostly spent inside numpy.
    - evolutionas 3y ago
      It does matter if you run your ML models in production. After we upgraded to 3.12 the average response time ~4-6ms decreased to stable sub 2ms. Latency decrease for p95 and p99 were even more significant
    - sgarland 3y ago
      There are plenty of web apps running in pure Python. It matters a great deal for them.
- dragonwriter 3y ago
  > most Python code in prod is what, 3.8? 3.8 is the oldest supported version, so I would hope not, but probably.
dataflow 3y ago
I imagine the way to do this is to start Python with some flag saying that it's in no-GIL mode. That way it's up to the user to decide if their libraries can handle it.
lisper 3y ago
There is a very simple practical solution: let the GIL (or something like it that forces single-threaded execution) be an option that you can turn on so that you can run broken legacy code.
- colordrops 3y ago
  Wouldn't the most practical solution be for a script or module to turn it off instead? Then you don't break any legacy code. Anyone writing code that is meant to work without the GIL would know to turn it off.
  - lisper 3y ago
    Sure, whatever. The point is that breaking legacy code doesn't have to be a show-stopper for eliminating the GIL, and backwards-compatibility does not have to be (indeed should not be) an overriding consideration in a GIL-free Python.
    - colordrops 3y ago
      What's the rationale? Making it opt-in avoids breaking legacy code, which is a huge advantage. What's the caae making it opt-out?
      - lisper 3y ago
        Why are you asking me to defend a position that I have explicitly disclaimed?
        deleted 3y ago
        [deleted]
        colordrops 3y ago
        No, you claimed the opposite. You suggest that disabling the GIL should be opt-out (or conversely enabling the GIL should be opt-in). This means that many legacy packages will break by default. You said it should be this way. Why? Don't downvote me, answer the question.
deleted 3y ago
[deleted]
mkoubaa 3y ago
Even with nogil, libraries can explicitly hold a global lock before any call. They just don't have to. I imagine some libraries will do that, and others will target performance. Users will vote with their tomls
pdonis 3y ago
If you have a multi-threaded program with data races, you already have a problem. The GIL does not mean no data races are possible. It just means that only one thread at a time can run Python bytecode. But the interpreter with the GIL can switch threads between bytecodes, and many Python operations, including built-in methods on built-in types that many people think of as "atomic", require multiple bytecodes. That's why Python already provides you with things like locks, mutexes, and semaphores, even though it currently has the GIL.
- rtpg 3y ago
  To put a finer point on this, I’ve had the misunderstanding in the past that the GIL made Python like JavaScript in some sense (only releasing the GIL on some explicit parts of code like sleep). But really Python threads can switch in the “middle” of your code. The reason the GIL is annoying is mostly performance related for Python code itself. My understanding is the GIL does not protect against Python-side bugs, and bugs from GIL removal would only be introduced from C extensions.
  - wokwokwok 3y ago
    ? Why do you think this? This has been discussed extensively in the past (1), and my understanding of the take away was that the GIL doesn't protect you from arbitrary execution order; it protects you from undefined behavior due to concurrent write/read in parallel scopes and the resulting data corruption. ...which, as I understand it, there is no specific reason it would be restricted to native extensions. Is there some more detail to the nogil proposal that addresses the type of UB you see in eg. c, with this? (Wouldn't that require that at some level the GIL still exists?) [1] - https://news.ycombinator.com/item?id=30420579 https://news.ycombinator.com/item?id=30420579
    - rfoo 3y ago
      What you said is precisely what nogil work is about. It's about replacing one global lock with finer grained synchronization primitives without much performance regression.
jacquesm 3y ago
The easiest way would be to have the GIL behind a feature flag that defaults to 'on'. That way you avoid yet another language split and if you don't want any possibly breaking changes you simply don't do anything at all. But if you want to run with the performance gains that a GIL free CPython would give you then you will have to do some extra testing to make sure that your stuff really is bullet proof with that flag set to the 'No-GIL' position.
perryizgr8 3y ago
> So much existing multithreaded Python code is written in an unsafe manner. Even multi-process Python code is often broken. The "recommended" way to serve a Django app is to run multiple workers (processes) using gunicorn. If you point the default logs to a file, even with log rotation enabled, all workers will keep stepping over each other because nobody knows which file to use. Keep in mind that this is broken by default, and this is the recommended way to use all this.
fulafel 3y ago
Good points. Re analyzability - it wouldn't have to be static analysis to be useful, you could do this with dynamic analysis.
usrbinbash 3y ago
> how are they going to do this while making sure programs are still correct? So much existing multithreaded Python code is written in an unsafe manner. a) It isn't a language maintainers job to make sure unsafe code written by someone else in that language runs correctly. b) The GIL doesn't prevent data races. It keeps the internal running of the interpreter sane, that's it's only job. There is a reason the threading library has a plethora of lock-classes

plonk 3y ago

I think the point was to let libraries declare whether they support nogil mode (opt-in), and your program would only run with no GIL if all the dependencies allow it? So they have all the time in the world to iron out those bugs.

eptcyka 3y ago
At what point can an interpreter establish that a given python script will not be importing any more modules?
- Borealid 3y ago
  If you import another module the gil could be re-enabled. Not saying that's what they will do, just what they could.
  - KMag 3y ago
    Right. It would be possible to implement the GIL as a readers-writer lock, where thread state includes a counter for the number of frames in the call stack that are within libraries not marked nogil. (Let's call these non-nogil libraries "GIL-dependent".) When the count goes from zero to one, the thread attempts to upgrade its reader lock to a writer lock. When its count goes from one to zero, the thread downgrades its lock from writer to reader. That way, there's at most one thread executing within GIL-dependent code at a time. Furthermore, if there is a thread executing within GIL-dependent code, all of the other threads are blocked waiting to acquire the GIL (in reader mode if they're nogil-safe, and writer mode if they're GIL-dependent.) As now, any thread holding the GIL in writer mode would need to drop the GIL when attempting to acquire any other lock (and re-acquire immediately afterward). To prevent starvation, one would presumably need a mechanism similar to periodic GC safepoints where nogil-safe threads still check if any thread is waiting to acquire the GIL in writer mode.
- lacker 3y ago
  Perhaps it could just fail at runtime if you ever import a module that doesn't support nogil mode? AFAICT that's how it works if, for example, you run Python code that uses f-strings in a Python version that doesn't support f-strings.
  - tyingq 3y ago
    Python's support for run-time version/flag/setting detection isn't great for this kind of thing either. With Perl, for example, you get a BEGIN {} block that is run before it tries to parse the script...so you can detect and shim, etc. Python bombs before you can gain any control, because it parses the whole file. So you can separate things into modules to get around that, but it's not great when you want a simple one-file script.
    - siddheshgunjal 3y ago
      We experienced programmers who know how to separate things into modules and use it wisely at the correct place. But most of the beginners and intermediate developers always tend to do it in a "Everything in single script" manner which might also make it difficult to have more control over the application's behaviour.
      - tyingq 3y ago
        You're making a lot of assumptions in a smug sort of way. There's plenty of spots where a single script makes sense, and plenty where it doesn't.
- Too 3y ago
  If that is the only way. They need to change some semantics around import statements to not be runtime conditional. (Mypy can do it without running the code so to some extent it is possible). With that in place, each module can at the top declare #nogilsafe and the interpreter can know that no more modules will be loaded runtime. Dynamic imports via importlib will need other consideration. Expecting every transitive module to add this marker is very optimistic though. It’s thousands of packages with hundreds of modules each, that need to add this everywhere. Other languages have done similar journeys, like typescript “strict” added at top of each file. Except those are a lot more local, by not expecting all dependencies to follow.
- plonk 3y ago
  I kind of thought Python would enable the GIL if any no-nogil libraries were installed, but even that is hard to define when you can modify sys.path with environment variables and code. Will look it up.
tgv 3y ago
Yes, but the plan is to remove the opt-in in time. That will put a lot of pressure on the eco system. I expect many libraries written in C or relying on C-based extensions to simply get dropped. Which will make that users will stay on the last GIL-supporting version. It's Python 2->3, but potentially worse.
- plonk 3y ago
  In my important dependencies, there are deep learning frameworks from billion-dollar companies, bindings for C++ libraries that are basically standard in the field, projects from CS labs with millions of users. I don't see any of them getting abandoned. But I guess I can't judge how much work removing the GIL could take. The big projects tend to be well-written and well-maintained, for what it's worth. Which projects do you have in mind that have a significant user base, are still maintained, and would be too costly to port for someone to do the effort?
  - tgv 3y ago
    The small ones, of which there are many more. Written for some specific purpose, half maintained, used in 1 or 100 projects. Those will suffer. Or perhaps worse, they get fixed, but wrongly, and introduce subtle bugs when your application is under a somewhat heavier load. Good luck finding the offenders. You might not even see a bug, only bad or irreproducible results. Multi-threading, concurrency, and parallelism are fraught with problems. Your precious ML/DL libraries may not even be upgraded, because writing neural network code is not the same as writing thread-safe code. If it comes from a CS lab, its authors have already left, and there's nothing worthy of a publication in adding thread-safety. Certainly not when you can simply stick to Python 3.last-gil-version.
    - plonk 3y ago
      > Your precious ML/DL libraries may not even be upgraded, because writing neural network code is not the same as writing thread-safe code. If it comes from a CS lab, its authors have already left, and there's nothing worthy of a publication in adding thread-safety. PyTorch and sklearn won't stop being maintained though. I don't rely on unmaintained research code in production, I adapt what I need under MIT license. Any other way sounds crazy. Plus, most research code is very high-level and uses the same facilities (from e.g. PyTorch again) that everyone else uses, the actual distributed and multithreaded work happens in the main libraries. You'll still be able to use the same neural network code that worked before. I don't see a huge problem for people who already had their dependency list under control. If you had anything that's both hard to replace and not big enough to be upgraded though, I'd argue that it was always going to bite you at some point.

bratao 3y ago

The GIL only protect the Interpreter. The only thing it may do is to make it infrequently. There are MANY threading bugs in actual Python code.

deleted 3y ago

[deleted]

KeplerBoy 3y ago

Just a little fun fact: The GIL absolutely does not prevent all race condition bugs. Threads contending for the GIL can already steal it from each other at unfavorable times and cause havoc.

ynik 3y ago
In fact, it prevents very few race condition bugs. Even inside a C extension where the Python API feels like it gives you control over when you release the GIL (with functions you'd have to call explicitly to release the GIL), it turns out that: * any operation that allocates new Python objects might trigger garbage collection * garbage collection may run `__del__` of objects completely unrelated to the currently running C code * `__del__` can be implemented in python, thus releasing the GIL between bytecode instructions Thus there's a lot of (rarely exercised) potential for concurrency even in C extensions that don't explicitly release the GIL themselves. nogil will make it easier to trigger data race bugs, but many of them will already have been theoretically possible before.

amelius 3y ago

On the other hand, perhaps translating existing modules to a No-GIL API is tedious but straightforward, and something that can be done using automated tools (perhaps even LLMs).

baggiponte 3y ago

I suppose no-GIL Python will be here in no less than 3/4 release cycles. 3.11 has been out for a year and most Python code in prod is what, 3.8? So I guess we won’t have to deal with this at scale before idk 2030 is approaching. I also don’t see Python runtimes in prod being updated from whatever they’re on now to newest releases. I don’t want to sound harsh, but the SC stated they don’t want to have another 2-to-3 migration, so people won’t update lightly. Yes, most of the content online right now might be dangerous to copy paste

sgarland 3y ago
Tangentially, 3.11 was the first release in quite some time to have major speed improvements across the board. The average is 25%, sometimes far more. Anyone who hasn’t upgraded to it by now is needlessly spending extra on compute.
- sapiogram 3y ago
  It's only 25% speedup in actual Python code, in domains like data science this won't matter much, because your cpu cycles are mostly spent inside numpy.
  - evolutionas 3y ago
    It does matter if you run your ML models in production. After we upgraded to 3.12 the average response time ~4-6ms decreased to stable sub 2ms. Latency decrease for p95 and p99 were even more significant
  - sgarland 3y ago
    There are plenty of web apps running in pure Python. It matters a great deal for them.
dragonwriter 3y ago
> most Python code in prod is what, 3.8? 3.8 is the oldest supported version, so I would hope not, but probably.

dataflow 3y ago

I imagine the way to do this is to start Python with some flag saying that it's in no-GIL mode. That way it's up to the user to decide if their libraries can handle it.

lisper 3y ago

There is a very simple practical solution: let the GIL (or something like it that forces single-threaded execution) be an option that you can turn on so that you can run broken legacy code.

colordrops 3y ago
Wouldn't the most practical solution be for a script or module to turn it off instead? Then you don't break any legacy code. Anyone writing code that is meant to work without the GIL would know to turn it off.
- lisper 3y ago
  Sure, whatever. The point is that breaking legacy code doesn't have to be a show-stopper for eliminating the GIL, and backwards-compatibility does not have to be (indeed should not be) an overriding consideration in a GIL-free Python.
  - colordrops 3y ago
    What's the rationale? Making it opt-in avoids breaking legacy code, which is a huge advantage. What's the caae making it opt-out?
    - lisper 3y ago
      Why are you asking me to defend a position that I have explicitly disclaimed?
      - deleted 3y ago
        [deleted]
      - colordrops 3y ago
        No, you claimed the opposite. You suggest that disabling the GIL should be opt-out (or conversely enabling the GIL should be opt-in). This means that many legacy packages will break by default. You said it should be this way. Why? Don't downvote me, answer the question.

deleted 3y ago

[deleted]

mkoubaa 3y ago

Even with nogil, libraries can explicitly hold a global lock before any call. They just don't have to. I imagine some libraries will do that, and others will target performance. Users will vote with their tomls

pdonis 3y ago

If you have a multi-threaded program with data races, you already have a problem. The GIL does not mean no data races are possible. It just means that only one thread at a time can run Python bytecode. But the interpreter with the GIL can switch threads between bytecodes, and many Python operations, including built-in methods on built-in types that many people think of as "atomic", require multiple bytecodes. That's why Python already provides you with things like locks, mutexes, and semaphores, even though it currently has the GIL.

rtpg 3y ago
To put a finer point on this, I’ve had the misunderstanding in the past that the GIL made Python like JavaScript in some sense (only releasing the GIL on some explicit parts of code like sleep). But really Python threads can switch in the “middle” of your code. The reason the GIL is annoying is mostly performance related for Python code itself. My understanding is the GIL does not protect against Python-side bugs, and bugs from GIL removal would only be introduced from C extensions.
- wokwokwok 3y ago
  ? Why do you think this? This has been discussed extensively in the past (1), and my understanding of the take away was that the GIL doesn't protect you from arbitrary execution order; it protects you from undefined behavior due to concurrent write/read in parallel scopes and the resulting data corruption. ...which, as I understand it, there is no specific reason it would be restricted to native extensions. Is there some more detail to the nogil proposal that addresses the type of UB you see in eg. c, with this? (Wouldn't that require that at some level the GIL still exists?) [1] - https://news.ycombinator.com/item?id=30420579 https://news.ycombinator.com/item?id=30420579
  - rfoo 3y ago
    What you said is precisely what nogil work is about. It's about replacing one global lock with finer grained synchronization primitives without much performance regression.

jacquesm 3y ago

The easiest way would be to have the GIL behind a feature flag that defaults to 'on'. That way you avoid yet another language split and if you don't want any possibly breaking changes you simply don't do anything at all. But if you want to run with the performance gains that a GIL free CPython would give you then you will have to do some extra testing to make sure that your stuff really is bullet proof with that flag set to the 'No-GIL' position.

perryizgr8 3y ago

> So much existing multithreaded Python code is written in an unsafe manner. Even multi-process Python code is often broken. The "recommended" way to serve a Django app is to run multiple workers (processes) using gunicorn. If you point the default logs to a file, even with log rotation enabled, all workers will keep stepping over each other because nobody knows which file to use. Keep in mind that this is broken by default, and this is the recommended way to use all this.

fulafel 3y ago

Good points. Re analyzability - it wouldn't have to be static analysis to be useful, you could do this with dynamic analysis.

usrbinbash 3y ago

> how are they going to do this while making sure programs are still correct? So much existing multithreaded Python code is written in an unsafe manner. a) It isn't a language maintainers job to make sure unsafe code written by someone else in that language runs correctly. b) The GIL doesn't prevent data races. It keeps the internal running of the interpreter sane, that's it's only job. There is a reason the threading library has a plethora of lock-classes