25 ms·
Python stands to lose its GIL, and gain a lot of speed
- TekMol 5y agoWhy is multithreaded performance important, what are the usecases where you cannot run multiple processes to spread your numbercrunching across CPUs? I am praying for CPython to become faster. But I need faster singlethreaded performance, so web applications benefit from it.
- kzrdude 5y agoInteractive use of Python: plotting, working with data, also would benefit from better multithreading. It's interactive, so it's (a bit) frustrating to wait for it to compute and see that it uses just one thread (the statistics ops are usually well threaded already, but plotting is not).
- chrisseaton 5y agoCommunicating between processes is more expensive than communicating between threads.
- TekMol 5y agoOk, but what is the use case where you need high bandwidth inter-thread/process communication?
- adgjlsfhk1 5y agomatrix multiply is an obvious one. partial differential equations are another. Sorting is one if you don't care about math.
- moron4hire 5y agoGames. Simulations. Backend servers for shared editor experiences. Teleconferencing.
- chrisseaton 5y agoExcept for embarrassingly parallel problems, the trade-off of generating more parallelism is usually needing finer-grained communication. Canonical examples in the literature are matrix multiplication, triangulation, and refinement.
- Spivak 5y agoLarge shared state is basically always the answer. You can cop-out and say use a database or Redis if that’s fast enough but that’s just making someone else use many threads with shared memory.
- VWWHFSfQ 5y agomultiple processes use a lot more memory than threads
- TekMol 5y agoCan you quantify that?
- redeyed 5y agoThere is such problem on Windows. When you create new process (spawn) it copies all memory of parent process.
- SkittyDog 5y agoYes, you can
- lazide 5y agoEven with copy-on-write? Have any actual numbers?
- SkittyDog 5y agoActual work is left as an exercise for the reader ;-)
- lazide 5y agoLast I did this, when the processes were fork()‘s of the parent (the typical way this was done), memory overhead was minimal compared to threads. A couple %. That was somewhat workload dependent however, if there is a lot of memory churn or data marshaling/unmarshalling happening as part of the workload, they’ll quickly diverge and you’ll burn a ton of CPU doing so. Typical ways around that include mmap’ng things or various types of shared memory IPC, but that is a lot of work.
- kaba0 5y agoWhat about context switches as well, much slower IPC, and basically “no native support”.
- rbjorklin 5y agoOne thing that comes to mind is for shared connection pools when you have thousands of workers connecting to the same stateful service.
- TekMol 5y agoWhich use case requires such a setup?
- lazide 5y agoMost databases (unfortunately), and since a ton of web apps use databases for state.. That said, many such databases are (or already have) rolled out connection pool proxies for reasons like this, so meh.
- lanstin 5y agoWeb front end backed by data base with expensive login, so you cache connections. Two tier architecture it would be called.
- Spivak 5y agoMySQL comes to mind. Unlike Postgres where connections are expensive MySQL encourages loading up the server with hundreds of simultaneous connections per server. Like anything, it’s possible to split this work out to a separate process but the IPC overhead is a lot.
- lostdog 5y agoHere's a use case: I was training a neutral net, and wanted to do some preprocessing (similar to image resizing, but without an existing C function). Inputs are batched, so the preprocessing is trivially parallelizable. I tried to multithread it in python, and got no speedup at all. That was a really sad moment, and I've never felt good about python since.
- isoprophlex 5y agoYeah, I had something similar. I wanted "as you wait for the GPU to churn through this batch, start reading the next batch from disk & preprocessing it on the CPU" Getting this to work turned out so ass backwards it made me sad Also I pity the fool who tries to connect a debugger to code using multiprocessing.Pool()...
- redeyed 5y ago> Also I pity the fool who tries to connect a debugger to code using multiprocessing.Pool() PyCharm works fine
- redeyed 5y agoYeah, had the exact problem. I solved it by using python threads and C function (cv2.resize)
- redeyed 5y agoI don't think it will happen soon. But what would be nice currently, is to create map reduce API w/o shared data or just readonly data. Something like ProcessPoolExecutor but instead of spawning new process and pickling input/output data, create threads w/o GIL w/o pickling input/output data.
- di 5y agoPrevious discussion: https://news.ycombinator.com/item?id=28880782 https://news.ycombinator.com/item?id=28880782
- Animats 5y agoYes. This is the discussion we had yesterday, but with more hype.
- np_tedious 5y ago> These changes are major enough that a fair number of existing Python libraries that work directly with Python’s internals (e.g., Cython) would need to be rewritten. But the cadence of Python’s release schedule just means such breaking changes would need to be made in a major point release instead of a minor one. Maybe time to rethink? https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/ https://www.techrepublic.com/article/programming-languages-w... If this is as promising as it sounds, it seems Python 4 now has its "thing" and is on the horizon. Or at least may become a serious thing to talk about
- dangerbird2 5y agoI'd wonder if it would be easier to introduce a totally new API along the lines of ruby's ractor API[1] that enables thread parallelism while keeping existing Thread behavior identical as with the GIL. Tons of python code relies on threaded code that is thread-safe under the GIL, but would completely blow up if the GIL was naively replaced. [1] https://docs.ruby-lang.org/en/master/doc/ractor_md.html https://docs.ruby-lang.org/en/master/doc/ractor_md.html
- arthurcolle 5y agoRactors don't offer very good performance yet. Better to have it be awesome right off the bat
- dangerbird2 5y agoYeah, That's what I thought. I think the greatest barrier now is that most multithreaded python code right now is just barely thread-safe, even with the GIL. I occasionally have to remind colleagues that even though the GIL guarantees instructions are atomic, you need to use mutexes and other synchronization primitives to ensure there is no race condition between multiple instructions. I'd imagine this change would be an optional interpreter feature initially, since removing the GIL would break the vast majority of code out in the wild, and it would be much more difficult to create an automated conversion tool like they did with the syntactic changes between 2.7 and 3
- randtrain34 5y agoDesign doc the proposer linked: https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...
- lvass 5y agoI have seen and written Python code that spawns various threads with shared mutable state. Is it possible that some day the same code would run in parallel? That could be a terrible (very) breaking change. I'm not against allowing in-process parallel execution but please let it require a new API.
- juanbyrge 5y agoTranslation: I have written buggy, racy software that has specific dependencies on thread timing. Please do not make significant improvements to Python because it will reveal these bugs in my software, and I will be forced to fix the bugs and use proper synchronization.
- lvass 5y agoIs that how you call things that have been working flawlessly and solving people's problems for over 10 years? Is needlessly breaking things that work an improvement to you?
- zelphirkalt 5y agoYou could probably simply lock the Python version you use for such code. No breakage there. If you must upgrade to a newer Python version, then you will have to repair broken code.
- doubled112 5y agoThis went really well for the Python 2 -> 3 upgrade
- lazide 5y agoIt did buy a decade or so (or more really) - not like the python2 distribution you downloaded and distribute with your program back then is going to get tracked down and shot in the head by Guido anytime soon. If you’re relying on whatever python version is distributed with whatever machine it happens to be on, there are a huge number of problems you’re already going to have.
- dleslie 5y agoI wonder how many folks think they're writing thread safe software with ease, and are unaware that they are leaning on the GIL? Could be that the impact of this change is far broader than just a few key libraries.
- deleted 5y ago[deleted]
- avianlyric 5y agoI don’t see how the GIL makes writing thread safe software any easier. The GIL might prevent two Python threads executing simultaneously, but it doesn’t change the fact that a Python thread can be preempted, meaning your global state can change at any point during execution without warning. Most of the issues with multi-threading come from concurrency, not parallelism. The GIL allows concurrency, you just don’t get any of the advantages of parallelism, which is normally the reason for putting up with the complexity concurrency creates.
- dleslie 5y ago> The GIL might prevent two Python threads executing simultaneously, but it doesn’t change the fact that a Python thread can be preempted, meaning your global state can change at any point during execution without warning. That thread behavior is enough to reduce the likelihood of races and collisions; particularly if the critical sections are narrow.
- laserlight 5y agoI wouldn’t call it thread-safe when race conditions are possible.
- jldugger 5y agoThen we need a term for when code race conditions are possible but rare enough that nobody using the software notices. thread-timebomb?
- mhh__ 5y agoWith all the effort that's been put into this how many people just jumped ship to a native/more-native language? I'm probably biased because I think Python is a hacked together mess but I just don't see what the point is in dragging it around.
- redeyed 5y agoAgain hammer to microscope comparison. Do you realize that your ignorance is bigger than Solar System?
- jgb1984 5y agoVery useful contribution, thanks! Mainly to demonstrate your own ignorance on the topic at hand.
- mhh__ 5y agoI did ask a question. The second part is me being belligerent but the question was sincere. I think Python is shit but so is most everything else, I'm interested in whether people jump ship or just work around it's issues at scale. I work on a programming language designed to avoid messy python scripts internally, so I am sincerely interested in these decisions.
- mixmastamyk 5y agoThe question is unanswerable. Python started as a scripting and prototyping language and that hasn't and won't completely change. It's fantastic at what it does from that perspective, late additions of complexity notwithstanding.
- ajkjk 5y agoThe difference is that it is so much easier, by orders of magnitude, to write code that gets shit done in python than any native language I'm aware of.
- Zababa 5y ago> With all the effort that's been put into this how many people just jumped ship to a native/more-native language? I think Go benefited a lot from that.
- Mikeb85 5y agoI use Ruby and not Python, but I think both have a lot of the same benefits and weaknesses. IMO, removing the GIL is a major mistake. The GIL is what allows you easy concurrency and to keep the language's 'magic' while ensuring correctness. If you need parallelism, there's processes and probably other tactics (I'm not super up to date on Python things). If you simply remove the GIL you have a bunch of race conditions, so you need a bunch of new language constructs, and it just adds a bunch of complexity to solve problems that don't really need solving. IMO they should just do what Ruby did with Ractors; basically a cheap alternative to spawning more processes. Rewriting absolutely everything that uses threads to be thread-safe is a waste of time.
- klyrs 5y agoIt's already easy to write race conditions in Python. if x in d: del d[x] else: d[x] = True Is a classic example -- if two threads execute that, you can't predict the outcome (but a KeyError is quite likely) The GIL only protects the CPython virtual machine; it doesn't protect user code. Concurrent code with shared mutable state already needs explicit mutexes.
- intrepidhero 5y agoWhat thread safe code can I write with the GIL that will have a race without it? I already have to be careful to only write to a shared object from one thread, since I have no guarantees on order of execution. The main benefit of the GIL, from my recent reading is that it makes ref counting fast and thread safe. The meat of the proposal is changing ref counting so that it's almost as fast and atomic without the GIL.
- ptx 5y agoWhat about setting a simple boolean flag, e.g. setting "cancelled = True" in the UI thread to cancel an operation in a background thread? In Java you would have to worry about safe publication to make the change visible to the other thread, but thanks to the GIL changes in Python are always (I think?) made visible to other threads.
- gigatexal 5y agoThese are the kinds of moves Python needs to stay relevant with amazing languages like Go out there that are just so simple yet powerful.
- sys_64738 5y agoKnowledge of the GIL forces a lot of python developers to not try to write multithreaded scripts. Doing away with it will make life harder for many folks, IMO. Have you tried explaining multithreading to some python scripters?
- aasasd 5y agoI mean, it stands to evolutionary reason that Python shouldn't have a GIL.
- aeturnum 5y agoI worked in Python for years and while I suppose I'm glad for any improvement, I have never understood the obsession with true multi-threading. Languages are about trade-offs and Python, again and again, chooses flexibility over performance. It's a good choice! You can see it in how widely Python is used and the diversity of its applications. Performance doesn't come from any one quality, but from the holistic goals at each level of the language. I think some of the most frustrating aspects from the history of Python have been when the team lost focus on why and how people used the language (i.e. the 2 -> 3 transition, though I have always loved 3). I hope that this is a sensible optimization and not an over-extension.
- lmilcin 5y agoThe issue is people try to get throughput from Python and GIL stands in the way. A lot of people learned Python and now want to use it for something more serious even if Python was not designed for it. I work on high volume backends (like million TPS on a single node) and Python for me is just a toy language. But I see no reason to purposefully inconvenience these people by keeping GIL if it can be removed.
- emerged 5y agoFrom my perspective as a huge Python fan, efficient multithreading is simply the only major thing missing from the language. I would still use C/C++/assembly for bleeding edge performance needs, but efficient multithreading in Python would have me reaching for alternatives far less often. Basically I love peanut butter ice cream (Python) I’d just like it even more with sprinkles.
- heavenlyblue 5y agoPython uses a lot of memory, multi-threading makes you have only one process sharing all the same memory.
- klyrs 5y agoLosing the GIL strictly makes the language strictly more flexible. Previous GILectomies tanked performance to an unacceptable degree. In single-threaded code, this one is a moderate performance improvement in some benchmarks, and a small detriment in others -- which is about as close to perfect as one could expect from such a change. That's why people are excited about it. At a higher level, Python is getting serious about performance. But this gives both flexibility and performance.
- ngrilly 5y agoImpressive proposal. That would remove a major limitation of CPython.
- ahmedfromtunis 5y agoIn my opinion, there's a time in a language's life were it should slow down the pace of "innovation". Code bases are complexe things and updating and upgrading them constantly just to keep up with the language maybe counterproductive. Python is there now, if you ask me. It should slow down and focus more on "maintenance" stuff with little to no impact on its interface. And maybe work on big projects like multithreading or stronger typing on the background and them when they're fully ready.
- pmlnr 5y ago> stronger typing Please, don't. There are wonderful strongly types languages out there, so if one wants or needs a strongly types language, use that, and not Python.
- smitty1e 5y agoThe gradual typing available now seems suitable. I have written plenty of code in typed contexts and plenty without. Python's "consenting adults" approach seems a win. Perhaps, without the GIL and with typing information included, additional performance gains with be on offer. But the "have it your way" nature of Python is a bigger win than either end of the data typing spectrum.
- pmlnr 5y ago> "have it your way" nature of Python TMTOWTDI - There's more than one way to do it - is Perl's motto :) Zen of Python suggests there's just one correct way.
- yellowapple 5y agoSoon Python will add sigils to variables and Satan will start having to wear mittens.
- smitty1e 5y agoThe typing module is much more recent than the Zen easter egg.
- vram22 5y agopinch.take(salt) Friend: I saw this interesting tech article ... Me: Site? Friend: mumble corp IT site mumble Me: Bye
- kzrdude 5y agoThe story of losing GIL is very popular in the news, and I like it too! .. but. Let's not count our chickens until they are home. I'm wondering if the Python dev community will take on this challenge. I hope so, Sam seems to really have put in a lot of effort!
- haihaibye 5y agoFYI the saying is "count our chickens until they are hatched"
- JacobHenner 5y agoSource: https://mail.python.org/archives/list/python-dev@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/#BJFDVRCZMEDOHEMCCIJJP6NTX6HOGC5L https://mail.python.org/archives/list/python-dev@python.org/...
- pvg 5y agoThe other day: https://news.ycombinator.com/item?id=28880782 https://news.ycombinator.com/item?id=28880782
- kjeetgill 5y agoAnyone know how far along graalpython is? I'd imagine it should be a suitable, if not superior, replacement with about as much effort right?
- didibus 5y agoBenchmarks seeing a 19.4x improvement going to 20 threads, an almost linear speedup. That's pretty amazing, in Java I feel I never manage to achieve linear speedup past a few threads. How are they managing the overhead?
- phkahler 5y agoAmdahl's law? It depends on the code. If you throw 100 cores at half of your code, that part may get 100x faster. But if the other half is single threaded you'll only see overall performance double.
- Offpics 5y agoIn other words, you probably never saw improvement past few threads, because your sequential code was slow.
- pca006132 5y agoWhat is the memory model for Python after removing the GIL? Will it introduce memory barriers and atomics with different memory ordering?
- biglost 5y agoPyston https://discord.gg/ARQ2WnB8 https://discord.gg/ARQ2WnB8
- cybernautique 5y agoI'm a random commenter with a chip on his shoulder. <grumble> Please escape Discord links so people don't accidentally click a live one and thereby expose data to Hammer & Chisel, Inc. </grumble> It's a constant slap-in-the-face how prevalent Discord is in the tech community. Need support with an obscure library? Discord. Want to talk to contributors for a project? Discord. I honestly don't know why people so greatly prefer Discord to IRC. The interface? Network effects? Whatever it is, Discord and H&C are disgusting, and I pray the tech community finds a way to escape it.
- LesZedCB 5y agothe interface and ubiquity, as you noted. can you fill me in? a couple google searches didn't turn up much.
- cybernautique 5y agoFill you in on what? I'd be happy to, I'm just missing context. Also, it's so disheartening to see people tout Discord's interface as its 'killer feature.' As far as I'm concerned, if I can't mold it into a decent TUI for my terminals, it's okay at best. If an app actively resists its interface being molded, however, that's just evil.
- fastball 5y agoPresumably GC wants to know why you hate Discord so much. You've spent a lot of words saying you don't like it, but haven't actually enumerated why.
- cybernautique 5y agoAh. That, I can do. Why I Hate Discord (A Manifesto) [Without Sources] Discord is aggressively proprietary. People ought to own their data, but Discord's architecture ensures everything gets hoovered into the mother-ship. At first, this was for nothing more than to facilitate their client-server communications model; recently, all of the hoovered data gets submitted to their AI moderation platform. (searched for sources on this but it was very hard to find anything. I remember talk about this ~c. Nov 2020, might be wrong) I should be able to modify an interface to my tastes. Modifying Discord is explicitly against their ToS, including their interface; attempting to do so will lead to a ban. Don't like their painted whore? Prefer to chat from your bespoke terminal? GTFO, Hammer & Chisel knows what's best for We Peons. Addenda: I find Hammer & Chisel developers and Discord admins to be disgusting. This is hearsay & personal opinion from my time hanging around the developer chats, but they all came across as nasty people. I, personally, believe many of the news reports surrounding the grooming controversies; searching "Discord admin controversy," "Discord allthefoxes controversy," "Discord cub policy," &c. turn up some relevant articles. Most of these articles are from low-quality reporting shops, but I buy in to the narrative.
- xbpx 5y agoI'm looking forward to something like the brilliant rayon parallel operations library https://github.com/rayon-rs/rayon https://github.com/rayon-rs/rayon We won't get data race free guarantees but if built into pandas or Vaex we can have a near transparent API. It'll really open things up for those apps running on 32 core machines. They're out there, I deploy these things frequently (Plotly Dash framework for large Enterprise customers).
- mkl95 5y ago> Multithreaded performance, on some benchmarks, scales almost linearly with each new thread in the best case—e.g., when using 20 threads, an 18.1× speedup on one benchmark and a 19.8× speedup on another. Interesting. When I write Python, and I write Python most of the time, I'm not chasing performance. But when it comes to speed, a 18x speedup gets Python up to par with currently much faster languages like Java. At least if you are willing to spam threads like your life depends on it.
- OneTimePetes 5y agoI do not understand the significance of this. If i want serious numbercrunch multi-thread performance in python, i execute it on the gpu? conda install numba & conda install cudatoolkit comes to mind? And you should data orientated, so all that remains from objects is arrays of structures with the index being the object indicator. Im honestly puzzled about the use-case..