12 ms·
gh-116167: Allow disabling the GIL
- helsinki 3y agoWhile the title is correct, it is a bit misleading, because disabling the GIL breaks the asyncio tests. It's like saying the engine can be removed from my car. Sure, it can, but the car won't work.
- TylerE 3y agoThis comment feels very disingenuous because non-threaded programs do in fact work.
- ollien 3y agoI mean, you're not wrong, but also it's a huge feat to provide a toggle for a major feature like the GIL. Though, if it's just asyncio that's broken, perhaps it's not like removing your engine, but rather your antilock brakes :) EDIT: > [the test synchronous programs] all seem to run fine, and very basic threaded programs work, sometimes Perhaps this is closer to removing the oil pan
- Kranar 3y agoWell this release will break any code that uses threads. The goal of this particular release is to work for thread-free programs.
- ollien 3y agoHow do single-threaded programs benefit from a lack of GIL?
- protomikron 3y agoThey don't.
- gtirloni 3y agoIt could remove the locking/unlocking operations.
- Retr0id 3y agoDoesn't removing the GIL imply adding back new, more granular locks?
- fiddlerwoaroof 3y agoSort of, but the biased reference counting scheme they’re using avoids a lot of locks for the common case.
- sapiogram 3y agoRemoving the GIL requires more locking/unlocking operations. For single-threaded program, it's a performance penalty on average: https://peps.python.org/pep-0703/#performance https://peps.python.org/pep-0703/#performance
- Kranar 3y agoThey don't benefit much from a lack of GIL, perhaps a small reduction in overhead. This feature is a first step towards being able to disable the GIL completely. It is intended to be implemented in a very conservative manner, bit by bit and so for this first step it should work for thread free code.
- kjqgqkejbfefn 3y agoDisabling the GIL can unlock true multi-core parallelism for multi-threaded programs, but this requires code to be restructured for safe concurrency, which isn't that difficult it seems: > When we found out about the “nogil” fork of Python it took a single person less than half a working day to adjust the codebase to use this fork and the results were astonishing. Now we can focus on data acquisition system development rather than fine-tuning data exchange algorithms. https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
- kjqgqkejbfefn 3y ago>We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck. To work around this problem, we sometimes use subprocesses, but in many cases the inter-process communication becomes too big of an overhead. To deal with the GIL, we usually end up translating large parts of our Python codebase into C++. This is undesirable because it makes the code less accessible to researchers.
- actionfromafar 3y agoMaybe they should look in to translating parts of their code base to Shedskin Python. It compiles (a subset of) Python to C++.
- logicchains 3y agoHow's it different from Cython, which compiles a subset of Python to C or C++?
- actionfromafar 3y agoShedskin has stricter typing, and about 10-100 times performance vs Cython.
- TylerE 3y agoSpeed. Admittedly not quite as much so the way this patch is implemented, since it just short circuits the extra function calls, doesn’t omit them entirely.
- 0cf8612b2e1e 3y agoRemoving the GIL results in slower execution. Without the guarantees of single thread action, the interpreter needs to utilize more locks under the hood.
- TylerE 3y agoNot in single threaded code.
- 0cf8612b2e1e 3y agoUmm, yes it does? For the longest time, Guido’s defense for the GIL was that all previous efforts resulted in an unacceptable hit to single threaded performance. Read PEP-703 (https://peps.python.org/pep-0703/#performance https://peps.python.org/pep-0703/#performance) where the performance hit is currently 5-8%
- TylerE 3y agoThat's to make it thread safe without the GIL. If you only care about single thread there's all kinds of stuff you can do.
- jononor 3y agoHow to ensure there are no other threads, confidently enough that one can turn thread safety off?
- Kranar 3y agoFrom when I was reading the proposal, the idea is that until a C extension is loaded, you can assume that there are no other threads. Then when a module is loaded, by default you assume that it uses threads but modules that are thread free can indicate that using a flag, so if a module indicates it's thread free then you continue running without the thread safety features.
- OskarS 3y agoReally, any code? I thought they were adding fine-grained locks to the python objects themselves? Are you saying that if I share a python list between two threads and modify it on one and read it on the other, I can segfault python?
- Kranar 3y agoWith this particular release, yes it will segfault. But down the road what you state is correct, this is just a first step towards that goal.
- thibaut_barrere 3y agoCouldn’t it work if each threads only touch thread-specific data structures?
- neilkk 3y agoThis isn't correct. TFA said that small threaded programs had been run successfully, but that the test suite broke in asyncio. Async I/O and threads are two different things, and either can be present in real code without the other.
- wolletd 3y ago"small threaded programs had been run successfully" I have ran a lot of programs containing race conditions successfully many times until I ran into an issue.
- Kranar 3y agoNot quite sure what your comment means exactly or how it implies what I said is incorrect. At any rate, test_asyncio contains a lot of tests that involve threads and specifically thread safety between coroutines and those tests fail. As far as async I/O and threads being distinct, I mean sure that is true of a lot of features but people mix features together and mixing asyncio with threads will not work with this particular release.
- neilkk 3y ago> this release will break any code that uses threads. > small threaded programs had been run successfully The second obviously contradicts the first, doesn't it? > mixing asyncio with threads will not work with this particular release. That's a very different claim to the first, and one that no longer conflicts with the second, isn't it?
- znpy 3y ago> While the title is correct, it is a bit misleading, because disabling the GIL breaks the asyncio tests. It's like saying the engine can be removed from my car. Sure, it can, but the car won't work. You're not supposed to drive a car that hasn't got out of the research and development laboratory either, so there's that.
- petters 3y agoYou also need to compile Python with a special flag activated. It’s not only an environment variable or a command line option.
- pharrington 3y agoBeing able to remove the engine from my car with the push of a button would be a pretty amazing feature!
- jagged-chisel 3y agoAnalogy breaking down and all, but … Only as long at it’s as easy to put back in
- deleted 3y ago[deleted]
- Retr0id 3y agoIs there a good overview of the bigger picture here?
- gtirloni 3y agohttps://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
- dang 3y agoRelated: Intent to approve PEP 703: making the GIL optional - https://news.ycombinator.com/item?id=36913328 https://news.ycombinator.com/item?id=36913328 - July 2023 (499 comments)
- rkagerer 3y agohttps://realpython.com/python-gil/ https://realpython.com/python-gil/
- protomikron 3y agoAlthough this is nice, the problems with the GIL are often blown out of proportion: people stating that you couldn't do efficient (compute-bounded) multi-processing, which was never the case as the `multiprocessing` module works just fine.
- kroolik 3y agoManaging processes is more annoying than threads, though. Incl. data passing and so forth.
- pillusmany 3y agoThe "ray" library makes running python code on multi core and clusters very easy.
- smcl 3y agoInteresting - looking at their homepage they seem to lean heavily into the idea that it's for optimising AI/ML work, not multi-process generally.
- pillusmany 3y agoYou can use just ray.core to do multi process. You can do whatever you want in the workers, I parse JSONs and write to sqlite files.
- kroolik 3y agoAlthough its great the library helps with multicore Python, the existence of such package shouldnt be an excuse not to improve the state of things in std python
- liuliu 3y ago`multiprocessing` works fine for serving HTTP requests or do some other subset of embarrassingly-parallel problems.
- dvas 3y agoExtra links for the no gil work for anyone else curious about this [0], [1]. [0] Multithreaded Python without the GIL https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0 https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD... [1] Github repo https://github.com/colesbury/nogil https://github.com/colesbury/nogil
- tentacleuno 3y agoFurther context on noGIL in general: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=nogil&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- bevekspldnw 3y ago[flagged]
- swtaarrs 3y agoThose links are both fairly old. See PEP 703 [0] and Sam’s nogil-3.12 repo [1] for more current versions. [0] https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/ [1] https://github.com/colesbury/nogil-3.12 https://github.com/colesbury/nogil-3.12
- Dowwie 3y agoWe are now one step closer to PythonOTP
- behnamoh 3y agoI've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked). I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal with numerous quirks later if I keep using Rust (like numerous string types). Is that something to be worried about if all I want from Rust is Python+Types+Concurrency?
- jondwillis 3y agoSwift or Kotlin might be what you’re looking for but nobody uses Swift for backend really, and I’m not sure about Kotlin.
- kuschku 3y agoKotlin has basically replaced java for many spring shops. It's really common in backend nowadays.
- imbusy111 3y agoSounds like you want Julia. It looks like Python, but also has, what you ask for. You can even run Python from Julia, so that alleviates the problem with a lack of libraries somewhat.
- deleted 3y ago[deleted]
- sparks1970 3y agoGolang? Pretty easy to pick up coming from Python and proper concurrency.
- scubbo 3y agoPython supports types! https://www.mypy-lang.org/ https://www.mypy-lang.org/
- Hugsun 3y agoIt will be very exciting to see how much faster they'll be able to make vanilla python. The value proposition is being challenged by the plethora of tools aiming to alleviate those issues. Speed improvers like Mojo, pytorch, triton, numba, taichi come to mind. There are so many different attempts at solving this problem that that the last time I wanted to try one of them, I found myself overwhelmed with options. I chose taichi which is pretty fun and easy to use, although somewhat limited in scope.
- nerpderp82 3y agoMojo should be viewed as an attack on the Python ecosystem due to it being a superset. It can consume Python, but it itself is not Python. Taichi is really underrated, it works across all platforms (including Metal), has tons of examples and the code is easy to write. And lastly, it integrates with the ecosystem and doesn't displace it. https://github.com/taichi-dev https://github.com/taichi-dev great demo reel of what Taichi can do, https://www.youtube.com/watch?v=oXRJoQGCYFg https://www.youtube.com/watch?v=oXRJoQGCYFg https://www.youtube.com/watch?v=WNh4Q7-OSJs https://www.youtube.com/watch?v=WNh4Q7-OSJs https://www.taichi-lang.org/ https://www.taichi-lang.org/
- objektif 3y agoNever heard of taichi before looks promising. Do you know any shop that uses it for prod code?
- fragmede 3y agoETH Zurich is using it for their physics sim courses, University of Utah is using it for simulations (SIGGRAPH 2022), OPPO (they make smart devices running Android), Kuaishou uses it for liquid and gas simulation on GPUs. Lots of GPU accelerated sim stuff. https://www.taichi-lang.org/ https://www.taichi-lang.org/ https://www.researchgate.net/publication/337118128_Taichi_a_language_for_high-performance_computation_on_spatially_sparse_data_structures https://www.researchgate.net/publication/337118128_Taichi_a_... https://github.com/taichi-dev/taichi https://github.com/taichi-dev/taichi
- misoukrane 3y agoFinally, looking forward to the benchmarks of many tools!
- xuhu 3y agoPEP-703 predicted in June 2023 an overhead of 15% when running with NoGIL: https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/9 https://discuss.python.org/t/pep-703-making-the-global-inter...
- rnmmrnm 3y agoMore than seeing it in main, I'm happy for the "python thread slow" meme officially going away now.
- scubbo 3y agoI wish I had your optimism. Thoughtless bandwagon-y "criticism" is extraordinarily persistent.
- samatman 3y agoThere's no need to pretend Python has virtues which it lacks. It's not a fast language. It's fast enough for many purposes, sure, but it isn't fast, and this work is unlikely to change that. Faster, sure, and that's great.
- fragmede 3y ago"Faster, sure" seems unnecessarily dismissive. That's the whole point of all this work.
- markhahn 3y agoYou seem to be implying that there is something inherently slow to Python. What? This topic is an example: a detail of one particular implementation, since GIL is definitely not inherent to the language. Just the usual worry about looseness of types?
- oivey 3y agoPython is inherently slow. That’s why people tend to rewrite bits that need high performance in C/C++. Removing the GIL is a massively welcome change, but it isn’t going to make C extensions go away.
- doctorpangloss 3y agoThere are worse hills to die on than this. But the Python ecosystem is very slow. It's a cultural thing. The biggest impact would be completely redoing package discovery. Not in some straightforward sense of "what if PyPi showed you a Performance Measurement?" No, that's symptomatic of the same problem: harebrained and simplistic stuff for the masses. But who's going to get rid of PyPi? Conda tried and it sucks, it doesn't change anything fundamental, they're too small and poor to matter. Meta should run its own package index and focus on setuptools. This is a decision PyTorch has already taken, maybe the most exciting package in Python today, and for all the headaches that decision causes, look: torch "won," it is high performance Python with a vibrant high performance ecosystem. These same problems exist in NPM too. It isn't an engineering or language problem. Poetry and Conda are not solutions, they're symptoms. There are already too many ideas. The ecosystem already has too much manic energy spread way too thinly. Golang has "fixed" this problem as well as it could for non-commercial communities.
- vlovich123 3y agoDoes anyone know why the biased reference counting approach described in https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/ just has a single thread affinity requiring atomic increments/decrements when accessed from a different thread? What I’ve seen other implementations do (e.g. various Rust crates implementing biased reference counting) is that you only increment atomically when moving to a new thread & then that thread does non-atomic increments/decrements until 0 is hit again and then an atomic decrement is done. Is it because it’s being retrofitted into an existing system where you have a single PyObject & can’t exchange to point to a new thread-local object?
- colesbury 3y agoWe could implement ownership transfer in CPython in the future, but it's a bit trickier. In Rust, "move" to transfer ownership is part of the language, but there isn't an equivalent in C or Python, so it's difficult to determine when to transfer ownership and which thread should be the new owner. We could use heuristics: we might give up or transfer ownership when putting an object in a queue.SimpleQueue, but even there it's hard to know ahead of time which thread will "get" the enqueued object. I think the performance benefit would also be small. Many objects are only accessed by a single thread, some objects are accessed by many threads, but few objects are exclusively accessed by one thread and then exclusively accessed by a different thread.
- vlovich123 3y agoI think you would do it on first access - “if new thread, increment atomic & exchange for a new object reference that has the local thread id affinity”. That way you don’t care about whether an object actually has thread affinity or not and you solve the “accessed by many threads” piece. But thanks for answering - I figured complexity was the reason a simpler choice was made to start with.
- orf 3y agoBut this would now make the reference count increment require a conditional? It’s a very hot path, and this would cause a slowdown for single-threaded Python code.
- karmasimida 3y agoThis is exciting, can't wait
- tommiegannert 3y agoFirst I read the news of tranched bread, and now this?! What a time! I was a bit disheartened when the Unladen Swallow project [1] fizzled out. Great to see Python back on the core optimization track. [1] https://en.wikipedia.org/wiki/CPython#Unladen_Swallow https://en.wikipedia.org/wiki/CPython#Unladen_Swallow
- fragmede 3y agotranched bread?
- KMnO4 3y agoI could be wrong, but I think it’s a clever alternative to the expression “best thing since sliced bread”.
- fragmede 3y agoahahah. I was thinking that it was a new python library or something that I hadn't heard of and was coming up short with Google. "tranced bread" is a fun name for some sort of library that breaks up files into pieces for better resilience for sending, like over BitTorrent.
- pas 3y agotranced bread sounds like what you go home with after a goa festival
- maest 3y agoGP is joking that this is the best thing since sliced (tranched) bread.
- oblio 3y agoOP is a subprime mortgage seller.
- deleted 3y ago[deleted]
- lbj 3y ago[flagged]
- dang 3y ago"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." "Don't be snarky." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- whalesalad 3y agoA great video tour of the GIL - https://www.youtube.com/watch?v=Obt-vMVdM8s https://www.youtube.com/watch?v=Obt-vMVdM8s
- deleted 3y ago[deleted]
- tiffanyh 3y agoELI5 I get in concept what the GIL is. But what's the impact of this change? Packages will now break, for the hope of better overall performance?
- nextaccountic 3y agoIf any package depends on the GIL, it will be enabled. Packages won't break
- hsbauauvhabzb 3y agoWhat packages might depend on the GIL and why would they need it?
- mlyle 3y agoPackages with native components that have not been updated for the semantics changes.
- whiterknight 3y agoAny C code that uses global variables casually.
- CJefferson 3y agoAlmost all packages depend on the GIL (at least at first). They assume they can "take the GIL", and then go and look at the various Python datastructures you passed them without worrying about them changing as they are being read. The later, when writing out their answer (which might involve editing something they were given), they can assume they are not changing. For example, you could write code which extends the length of a list to 100, then fill in all the members, not worrying half way through your loop another thread shrinks the list back down.
- imtringued 3y agoThis only applies to native extensions.
- pgraf 3y agoIf anyone wondered GIL = Global Interpreter Lock
- purpleidea 3y agoIt's neat that this will eventually improve some python code, but at the end of the day, it's still a badly typed language, which will still be slower and less safe than more modern languages. Learn golang or rust instead. Impressive that they are managing this though!
- Cacti 3y agobut Python has my libraries, and go and rust do not. >> 99% of my compute is offloaded to compiled BLAS or CUDA. types are enough. and memory and type safety are not terribly relevant for my use cases.
- alfalfasprout 3y agoBad take. One doesn't just migrate massive ecosystems to "go or rust". Python, like it or not, is the lingua franca of ML/AI and science. Worse, as a user of Go/Rust it's very disingenuous to expect the same kind of iteration ability in those languages as you get in Python. With tools like pyright now + the work on nogil everyone benefits from this using Python.
- takeda 3y agoRust, sure, but typing in golang isn't superior to python's type annotations. Also Rust became a quite popular tool for python extensions, where you can offload performance to rust and business logic to python.
- jen20 3y ago> typing in golang isn't superior to python's type annotations. Yes it is, in every way. When the first job of choosing to use types is choosing which of the five or so type checkers, each deficient in their own way and incapable of dealing with the poor idioms the language encourages to proliferate, you know you're on the wrong path.
- AdamJacobMuller 3y ago> Learn golang or rust instead. Bad take. Learn golang and rust and python. You should use the language which is suited to the task, sometimes that's golang sometimes that's python and sometimes it's rust. It's impressive that the python team as a whole continues to improve in such big ways after more than 30 years of development. It's more impressive that the python team managed to navigate 2to3 and come out stronger.