14 ms·
Progress on No-GIL CPython
- qeternity 3y agoI find the current focus on GIL-less Python really strange. The Faster cPython team set an ambitious goal of increasing cPython performance 50% with each release. 3.11 contained some real improvements, but nowhere near 50%. And for much of our testing, 3.12 is either flat or slower. True multi-threading would be great, but I would much rather have improved single threaded performance first. I of course respect that our needs may not represent everyone else's, and we are grateful for all the work that has gone into making Python a great language. But what am I missing?
- benrutter 3y agoMy understanding is they are two seperate projects within cpython and don't necessarily have the same people working on them. I agree with you that if it was one or the other, most use cases are gonna be better met with straight up faster single threaded code. But why not have both?
- slashdev 3y agoBoth is not really on the table. Getting rid of the Gil will slow down single threaded code.
- rowanG077 3y agoWhy would removing the GIL slow down single threaded code?
- edgyquant 3y agoI believe without a GIL objects need to be thread safe which comes at a price
- viraptor 3y agoYou still need to run the same memory management/accounting for all the objects as when you run multithreaded, because at any point a new thread may be started. And it takes more time to make all MT access safe then to prevent other threads from running.
- Austizzle 3y agoAs far as I know, in order to make everything thread safe in the same way the gil did, they need to add locks in a lot of places to make sure that no two objects can be modified at the same time. Adding those locks will slow down execution
- Jtsummers 3y agoNo-GIL forces the addition of locks and other concurrency controls into the execution path of single-threaded code that's not there now. It's the same kind of hit you get going from thread-unsafe code (hopefully intended for single-threaded execution) to thread-safe in any other project (since, well, it's the same; the GIL means they could write code with single-threaded assumptions). Checking and taking locks is not a free action.
- rowanG077 3y agoI thought the point of No-GIL was to force users of python to provide proper locking.
- Jtsummers 3y agoTo achieve no-GIL, libraries and extensions along with the underlying language implementation will need to add in the proper locking. For users, it'll be mixed. Users writing single-threaded code shouldn't have to change a single line of code, but they'll see (sans the concurrent efforts to speed up the underlying implementation) slower performance due to everything done to achieve no-GIL (the actual net effect will be a performance boost due to that concurrent effort over time). If they're writing multi-threaded code, then they should be writing it to be threadsafe now, not assuming that the GIL will protect them (because it doesn't guarantee it now anyways). So nothing should actually change for most users of Python if they're writing correct code today.
- kortex 3y agoPython's reference counting garbage collector would also need concurrency controls.
- Jtsummers 3y agoThe last numbers I saw put the performance penalty of no-GIL at under 10%, so both are still on the table since Python has a lot of single-threaded performance left to recover. You can get both a much faster-than-current Python in single-threaded and real threads.
- alfalfasprout 3y agoYep and when the faster cpython team gets to implementing advanced JITing (which GVR has said is planned) that'll be huge.
- slashdev 3y agoObviously you can recover it, but they will Be a performance hit. Single threaded without GIL will always be slower than with it. I’ll be surprised if they can keep it to only 10%. It will depend on the workload and there will Be some pathological cases.
- alfalfasprout 3y ago> Both is not really on the table Incorrect. > Getting rid of the Gil will slow down single threaded code Correct. But that's before fastercpython improves single threaded performance closing that gap if not reversing it.
- softwaredoug 3y agoThe computationally expensive parts of “Python” are done in libraries like numpy, tensorflow, etc. Python makes it really handy to play with low level abstractions in a higher level language. So yeah I’ve never stressed much about the GIL as a long time pythons dev
- jhj 3y agoYes but coordinating the computationally expensive parts usually needs to be done across multiple CPU threads. Typically one should feed each GPU from a different CPU thread, even in native C/C++ land, as in many cases the kernels being run on the GPU may not be entirely heavy compute but instead on the order of the kernel launch overhead from the host (1-4 us); not every kernel run would be orders of magnitude longer runtime than the overheads involved. Data loaders and other non-computationally expensive parts (I/O latency or throughput bound things) need to be done as well, which make sense to multi-thread as well.
- deleted 3y ago[deleted]
- mardifoufs 3y ago> The computationally expensive parts of “Python” are done in libraries like numpy, tensorflow, etc. Python makes it really handy to play with low level abstractions in a higher level language. > So yeah I’ve never stressed much about the GIL as a long time pythons dev You almost always have to use python objects at some point or another, even with very high perf libraries. Unless you are using python as nothing else than glue code, and loading and preprocessing the data entirely outside of your python code. At some point (and I'm not talking about extreme scales here, just feeding a single datqaloader can be enough) you hit a very hard performance bottleneck. Sure you can just use more external code at that point but I think the intention is to make python at least more suitable for the non compute intensive stuff. Like, it is already suitable right now but it's not getting better, while everything around it is (I'm not talking about other languages, what I mean is that the tools and libraries are getting better so without improvement to the language the bottleneck will just get worse)
- fulafel 3y ago
- bratao 3y agoIn my opinion, Python needs to have an urgent answer to the use of multiple cores. AMD just launched a CPU with 96 cores. Today the use on multiple cores is through the use multiprocess, which has many limitations. I understand that the multiple interpreters could come with something like Goroutines, but I still like the real multi thread option more.
- brianwawok 3y agoMy server can easily fill all 96 running standard Python under gunicorn.
- NegativeLatency 3y agoand if you use and async framework it's even more capable
- iterateoften 3y agogunicorn uses multiple processes
- aeyes 3y agoYou can still share most of the memory if you load everything before the fork so why is having multiple processes a problem?
- euiq 3y agoI think you're missing that your needs are different from those of the people quoted in PEP 703: <https://peps.python.org/pep-0703/#motivation https://peps.python.org/pep-0703/#motivation>
- qeternity 3y agoI literally said I understand that my needs are different. I get why the PEP exists. I don't get why it's receiving such priority.
- rvanlaar 3y agoThere's a GIL presentation [1] from 2010 which shows Python 3.2 on a 1 CPU machine was faster than on a dual core. In all, there might be reason to expect GILless python to be faster single core in certain scenario's. [1] https://youtu.be/Obt-vMVdM8s?t=2047 https://youtu.be/Obt-vMVdM8s?t=2047
- alfalfasprout 3y agoThey're completely different goals. Yeah in theory multithreaded python lets you speed up certain programs... but the way in which it does so is important. With nogil python you can effectively have multiple threads that eg; call out to C code while having shared state accessible as python objects. This is pretty key for ML-- in fact this current incarnation of the PEP came from the PyTorch team. Single threaded performance is important too but there have already been lots of decent workarounds for critical sections (eg; numba, Cython, and now things like Mojo). The ordering is important too-- a lot of the faster cpython work would be thrown away completely if nogil came about. So the teams have had to coordinate. In the ideal world that means both nogil mode + improvements to single threaded performance (Guido even hinted sophisticated JITing is being considered).
- deleted 3y ago[deleted]
- malcolmgreaves 3y agoI always openly wonder with this proposal — how are they going to do this while making sure programs are still correct? So much existing multithreaded Python code is written in an unsafe manner. Specifically, talking about data races I’ve seen time and again in codebases across companies and OSS projects. The programs don’t break only because they implicitly rely on the GIL providing execution to a single thread at a time. If the GIL is gone, then these programs will break. And since Python is such a dynamically typed language, I seriously doubt that there exists a static analyzer that could identify these issues in existing Python programs. More likely, they’ll be insidious bugs that crop up at runtime in a non-deterministic fashion. Ideally leading to a crash, With this class of bugs, it’s likely to just result in incorrect operations being performed. Perhaps this GIL-less proposal isn’t actually intended to be used on the overwhelming majority of programs? Maybe it’s just a hyper specialized tool for a very few number of circumstances where the programmer knows, there won’t be a GIL, and can program against that fact?
- plonk 3y agoI think the point was to let libraries declare whether they support nogil mode (opt-in), and your program would only run with no GIL if all the dependencies allow it? So they have all the time in the world to iron out those bugs.
- eptcyka 3y agoAt what point can an interpreter establish that a given python script will not be importing any more modules?
- Borealid 3y agoIf you import another module the gil could be re-enabled. Not saying that's what they will do, just what they could.
- KMag 3y agoRight. It would be possible to implement the GIL as a readers-writer lock, where thread state includes a counter for the number of frames in the call stack that are within libraries not marked nogil. (Let's call these non-nogil libraries "GIL-dependent".) When the count goes from zero to one, the thread attempts to upgrade its reader lock to a writer lock. When its count goes from one to zero, the thread downgrades its lock from writer to reader. That way, there's at most one thread executing within GIL-dependent code at a time. Furthermore, if there is a thread executing within GIL-dependent code, all of the other threads are blocked waiting to acquire the GIL (in reader mode if they're nogil-safe, and writer mode if they're GIL-dependent.) As now, any thread holding the GIL in writer mode would need to drop the GIL when attempting to acquire any other lock (and re-acquire immediately afterward). To prevent starvation, one would presumably need a mechanism similar to periodic GC safepoints where nogil-safe threads still check if any thread is waiting to acquire the GIL in writer mode.
- yodsanklai 3y agoDidn't OCaml undergo a similar evolution? is there anything comparable between these two projects?
- debugnik 3y agoI don't think so. Rather than removing the global lock and breaking existing code, OCaml 5 introduces a new primitive called a "domain" which manages one or more threads with a shared lock. So the existing threads API spawns threads in the current domain, which lets you isolate code that expects to take the lock, while new code can spawn new domains starting with one thread instead. You can also use both deliberately as a form of scheduling. Python instead is trying to make the lock entirely optional, globally and outside the control of library writers. However, I think the Python lock is only guaranteed to protect the runtime itself, so most code depending on it is probably buggy anyway, so I think their plan is viable. The only thing they may have in common is having to scan the entire codebase of their runtimes for unexpected shared state and fix that, as well as revising their C ABIs.
- bfrog 3y agoIt seems like every 5 years I hear about how someone is going to make python not have a GIL, improve performance drastically, etc etc. Then reality comes in and says, "nah". Honestly at some point its better to likely start porting code to a language where you can control parallelism better. There's a few nice modern options that don't involve all the headaches of C++ or C. Could even be extensions as has been the case since the dawn of time.
- voz_ 3y agoBad faith take.
- kmod 3y agoIgnoring the pessimism for a moment But this nogil version is the first time we have an actually working GIL removal. All of the other ones were incomplete to the point of being non starters, and mostly served as discussion material. This is an actually working implementation which deals with the subtle issues that the other projects didn't even get to, and has gotten to the point that it's a technical possibility to commit it to main (though obviously with a huge migration to think about). So in this sense this is a very different discussion than all of the previous discussions about the GIL
- notatallshaw 3y agoThe difference between this attempt and previous attempts is: * The PEP has been announced to be accepted (Steering Council are still working on details for final wording) * Many things are already landing on CPython main in preparation for this Unless something absolutely show stopping comes up in the next 12 months it will almost certainly released in 3.13 or 3.14 as an optional compiler flag.
- Affric 3y agoInteresting discussion there too. With modern computers I wonder if explicit parallelism is more fundamental to what our computer science will be than is in vogue in textbooks. Perhaps we should always be writing explicitly parallel code at this point.
- Gh0stRAT 3y agoHumans are bad at reasoning about multiple threads simultaneously, so I suspect the more practical shift is the trend we've already been seeing toward more declarative syntax. eg `for` loops are being replaced by `foreach` loops,`map` and `filter` operations, etc. These tell the compiler/interpreter that you want to do some operation to all the items in your datastructure, leaving it up to the compiler/runtime whether and how to parallelize the work for you.
- MR4D 3y agoI would upvote this 100 times if I could. I've thought this way ever since MacOS added the Grand Central Dispatch [1]. Of course, I thought industry would follow quickly and that tooling would coalesce around this concept pretty quickly. Seems the industry wants to take its sweet time. [1] - https://en.wikipedia.org/wiki/Grand_Central_Dispatch https://en.wikipedia.org/wiki/Grand_Central_Dispatch
- cpgxiii 3y agoI mean, OpenMP dates back to 1997 (1998 for C and C++). Apple, however, has never supported it for what can only be selfish reasons (particularly since Clang has a quite good implementation provided by Intel, which can easily be installed on a Mac if you want). GCD came a decade later. For basic parallelism, nothing beats OpenMP for ease of adapting existing code (often a single "#pragma omp parallel for" directive is enough). Even for more complex parallelism, particularly where per-thread resources need to be managed, OpenMP still provides a much simpler programming model than the alternatives.
- Jtsummers 3y ago
- mirekrusin 3y agoUse -ng (no-gil or next-generation).
- KaiserPro 3y agogil-sans always good to have a pun.
- fireflash38 3y agoOr python-lungs. Cause it doesn't have gills anymore?
- gregw2 3y agoI think if we want a python with no gil(l)s we should call that version “lamprey”. Sharper teeth in that version. Same shaped creature though!
- twoodfin 3y agoHaving intense flashbacks to the great wave of Unix threading support, where exactly what you had to do as a developer varied massively from platform to platform: New compiler flags, new linker flags, linking in different libraries, using an entirely different compiler command (looking at you, AIX!) …
- parhamn 3y agoHindsight is 20/20 but if the Python folks knew how long and bad the 2 to 3 transition would be, they probably would've committed to a much bigger facelift on the interpreter internals too. 12 year transition and single threaded performance is still abysmal and it has a few painful transitions left to get to real multi-threading. As kind as one should be with opensource development, at some point is it fair to call it a very poorly managed language?
- lacker 3y agoNah, it's not poorly managed. Python has a lot of problems. But they are all problems that stem from Python's success. The worst parts of Python are the parts that are hard to change because Python is so popular, so the ecosystem is really large, so every sort of change becomes harder due to backward incompatibility.
- tyree731 3y agoOne of Python’s most serious issues is that it is one of the slowest programming languages actively developed and used today, if it isn’t the slowest outright. This isn’t due to its success, this is due to the language’s performance having been deemed not a priority until somewhat recently.
- Daishiman 3y agoNo, this is definitely a key to its success. A simple C implementation allows everyone and their mother to hack at it and interface with C libraries and add try features and evolve the language through endless PEPs. Compare Python's language evolution to Java's abysmally slow language evolution because every new feature has to be implemented in a way that works with the JVM's JIT-compatible speed hacks. A _ton_ of very useful things Java could have done simply cannot be done because you can't work against the grain of the JVM. If Python's backward-compatibility is a pain, you have no idea how bad it is in the JVM (see how all the JVM's caveats have hobbled the semantics of generics as a good example). Not providing the end users any guarantees about performant code means that coders offload the performant areas to libraries that use languages built for performance, keeping that complexity outside of the interpreter and language.
- xwowsersx 3y agoAlways good to get updates on this. Question for folks here: what applications/services would you be writing if No-GIL were complete today? Or are there existing applications you have that would benefit from this? Just curious what people are anticipating exactly.
- deleted 3y ago[deleted]
- logicchains 3y agoPyTorch code using awful to reason about multiprocessing code for data loading could be rewritten to something much more sane with threads. There'd be no need to do all kinds of ugly hacks to avoid unnecessary copying of large arrays between threads, as they could all simply share the same (read only) memory.
- facu17y 3y agoI'd rather port my Python code to Mojo and get, multi-threading, SIMD and other speedups
- csjh 3y agoThat would be nice in a world where Mojo is more complete but it is nowhere near that level right now
- Too 3y agoTo be fair, neither is nogil.
- jeremycarter 3y agoAgree. I'd rather just rewrite in Rust, Nim and .NET.
- deleted 3y ago[deleted]
- facu17y 3y agoThis downvoting to -3 is illustrative of how HN down votes are about territorial warfare, ego, etc. Has nothing to do with logic. You don't like to port your python code to nogil python? I'm gonna downvote you. WTF is wrong with you people.
- Daishiman 3y agoThis got a downvote from me because, as a poster of over a decade, I've seen people complain similarly with Java, C#, PHP, C, and every other popular language under the sun where the apparent solution is to move over to a niche language nobody knows and nobody cares about. It's immature logic from people that lack a sense of perspective of what engineering compromises are and the fact that even substantial language warts don't move the needle as far as justifying language switches. If you know about Python performance and what's the most you can squeeze from the language, its libraries implemented in C for high performance, and other things in the ecosystem, you already knew if the code you're writing was good for Python or not. If it was, between Cython, C bindings, and NumPy you should be covered. If not, you can drop down to C, Rust and C++ libraries and the slow performance of the glue code is an afterthought. It is the opinion of an overwhelming majority of coders, who need high-performance code for maybe 5% of what they write (and most of that being code getting run relatively very little), that higher performance is not as high a priority for those users as some people think.
- c-smile 3y agoKeep GIL but introduce Workers a la JS [0] ? Worker [thread] is a sandboxed VM [context] with its own GIL. Communication between these is done through messaging so no sync primitives are required. Go's routines have similar concept. I think that if it OK for Go it should be OK for Python, no? [0] https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
- dragonwriter 3y agoThat's subinterpreters in a single process, and the Python runtime already has that (though the only API is from C, not Python, currently); Python API is planned for 3.13.
- fulafel 3y agoYep. The traditional sub-interpreters system in Python still used the single shared GIL so it wasn't a solution for parallelism. But just recently the "per-interpreter GIL" was implemented, it shipped in Python 3.12.
- ksec 3y agoSounds like Ruby's Ractor.
- thibaut_barrere 3y agoExactly!
- akdor1154 3y agoGo also has very easy data races when accessing the same data from different goroutines, I'm not sure Python would accept such a well-cocked footgun.
- tgv 3y agoNo-gil has this plus a bigger footgun.
- VagabundoP 3y agoThe shebang issue should probably lean on existing Python conventions: from future import nogil It would hot swap interpreters at that point.
- ikari_pl 3y agoit could be a nightmare to implement if it's not the first module, and then the first import, to execute
- pletnes 3y agoNot sure about anything nogil related, but future imports already have to be the first executable statement in a file.
- hsfzxjy 3y ago`from __future__ import ` is a specialized statement to indicate flags rather than a runtime statement. https://docs.python.org/3/reference/simple_stmts.html#future-statements https://docs.python.org/3/reference/simple_stmts.html#future...
- kzrdude 3y ago"""A future statement is a directive to the compiler that a particular module should be compiled using syntax or semantics that will be available in a specified future release of Python where the feature becomes standard.""" future statements are module specific and GIL/no-GIL doesn't fit easily into that model.
- IshKebab 3y ago> The gil version of python executable is "python3". Not on Windows!
- deleted 3y ago[deleted]
- DanielHB 3y agoI am not an expert in python but I feel that the JS model where everything is on the event loop and there is no actual threads seem better for a dynamic language. A lot of parallelization can be achieved with web-workers, but of course at the cost of relying on copying memory between workers (ie no shared memory). There have been some proposals to add full shared memory constructs (SharedArrayBuffer) and synchronization (Atomics) mechanisms, but they are special constructs and don't work with normal javascript objects. Quite limited but provide full parallelism for things that usually need it (buffers). One thing people often forget is that thread-safe data structures are usually a lot slower than single-threaded one, everything in JS is single threaded and in the event loop, but if you really need it there are some scape hatches. I don't know, this just feels better and simpler? If you really need to you can go down to a lower level language for full memory sharing data structures.
- logicchains 3y ago>I am not an expert in python but I feel that the JS model where everything is on the event loop and there is no actual threads seem better for a dynamic language Python has become essentially the primary language for scientific computing and deep learning, and for this the JavaScript approach is absolutely inadequate. Real shared memory parallelism is needed to avoid the unnecessary copying that the subprocess approach entails. >I don't know, this just feels better and simpler? If you really need to you can go down to a lower level language for full memory sharing data structures A low level language is already used in Numpy, PyTorch etc., but that's not enough; the Python glue code becomes unnecessarily painful when using multiprocessing, pain that would go away with proper threads.
- DanielHB 3y agoIf you disregard the existing libraries and think purely about the concepts and powers of the languages, the JS model seems better to me. You can avoid memory copying between workers by using SharedArrayBuffer
- danielrhodes 3y agoI don't think people use Python because it's fast. It gained momentum because it has good libraries for data science and ML. Other languages, including JS, are far faster at this point. However, the reason to use JS in particular is because when you have a lot of I/O blocking calls (DB calls, web requests) it really is a lot faster. But if you do anything with numbers, JS is the wrong language - it's so limited here that it's almost laughable. Same with dates. That makes it a very bad choice for data science.
- dcow 3y agoRe naming: python4 python3-gilfoil python3-gilfree
- cmacleod4 3y agoSo Python now has a chance of catching up to Tcl for multithreaded performance - see https://www.hammerdb.com/blog/uncategorized/why-tcl-is-700-faster-than-python-for-database-benchmarking/ https://www.hammerdb.com/blog/uncategorized/why-tcl-is-700-f... :-)