19 ms·
Really excited for this. Once some more time goes by and the most important python libraries update to support no GIL, there is just a tremendous amount of perf
by eigenvalue 2y ago
Really excited for this. Once some more time goes by and the most important python libraries update to support no GIL, there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects. It's also a good opportunity for new and more actively maintained projects to take market share from older and more established libraries if the older libraries don't take making these changes seriously and finish them in a timely manner. It's going to be amazing to saturate all the cores on a big machine using simple threads instead of dealing with the massive overhead and complexity and bugs of using something like multiprocessing.
- phkahler 2y agoI feel like most things that will benefit from moving to multiple cores for performance should probably not be written in Python. OTH "most" is not "all" so it's gonna be awesome for some.
- MBCook 2y agoBut it would give you more headroom before rewriting for performance would make sense right? That alone could be beneficial to a lot of people.
- rty32 2y agoI think it is beneficial to some people, but not a lot. My guess is that most Python users (from beginners to advanced users, including many professional data scientists) have never heard of GIL or thought of doing any parallelization in Python. Code that needs performance and would benefit from multithreading, usually written by professional software engineers, likely isn't written in Python in the first place. It would make sense for projects that can benefit from disabling GIL without a ton of changes. Remember it is not trivial to update single threaded code to use multithreading correctly. in Python language specifically. Their library may have already done some form of parallelization under the hood
- bdd8f1df777b 2y ago> Code that needs performance and would benefit from multithreading, usually written by professional software engineers, likely isn't written in Python in the first place. There are a lot of simple cases where multi-threading can easily triple or quadruple the performance.
- Certhas 2y agoBut multiprocessing can't? I used to write a ton of MPI based parallel python. It's pretty straightforward. But one could easily imagine trying to improve the multiprocessing ergonomics rather than introducing threading. Obviously the people who made the choice to push forward with this are aware of these options, too. Still mildly puzzling to me why threads for Python are needed/reasonable.
- pdhborges 2y agoIt doesn't to be puzzling just read the motivation section of https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
- Certhas 2y agoI know that document. It doesn't really answer this point though. It motivates the need for parallelism in slow python by noting that this is important once other performance critical code is in extensions. But the mai point against multiprocessing seems to be that spawning new processes is slow ... That single "alternatives" paragraph doesn't answer at all why mp isn't viable for python level parallelism. I am no longer invested in python heavily, I am sure there are discussions or documents somewhere that go into this more. Might be that it's simply that everyone is used to threads so you should support it for sheer familiarity. All I am saying is it's not obvious to a casual observer.
- bdd8f1df777b 2y agoA common pain point for me is parallel data loading and processing for PyTorch or TensorFlow. Multiprocessing has a lot of bugs and pain points to deal with when doing ML. Examples: https://github.com/search?q=repo%3Apytorch%2Fpytorch+multiprocessing&type=issues https://github.com/search?q=repo%3Apytorch%2Fpytorch+multipr.... Most of these issues do not exist in a multithreading world, because resource sharing is trivial in that case. Since Python leads over any other languages in the ML community, and ML is a hot topic right now, it makes sense for Python developers to secure the lead by making the life of ML developers easier, which is by introducing GIL-less multi-threading.
- DanielVZ 2y agoUsually performance critical code is written in cpp, fortran, etc, and then wrapped in libraries for Python. Python still has a use case for glue code.
- andmkl 2y agoYes, but then extensions can already release the GIL and use the simple and industrial strength std::thread, which is orders of magnitude easier to debug.
- woodruffw 2y agoConcurrent operations exist at all levels of the software stack. Just because native extensions might want to release the GIL and use OS threads doesn't mean pure Python can't also want (or need) that. (And as a side note: I have never, in around a decade of writing C++, heard std::thread described as "easy to debug.")
- ipsod 2y agoReally? Cool. I expected that dropping down to C/C++ would be a large jump in difficulty and quantity of code, but I've found it isn't, and the dev experience isn't entirely worse, as, for example, in-editor code-intelligence is rock solid and very fast in every corner of my code and the libraries I'm using. If anyone could benefit from speeding up some Python code, I'd highly recommend installing cppyy and giving it a try.
- wongarsu 2y agoI often reach for python multiprocessing for code that will run $singleDigit number of times but is annoyingly slow when run sequentially. I could never justify the additional development time for using a more performant language, but I can easily justify spending 5-10 minutes making the embarrassingly parallel stuff execute in parallel.
- throwaway81523 2y agoI've generally been able to deal with embarassing parallelism by just chopping up the input and running multiple processes with GNU Parallel. I haven't needed the multiprocessing module or free threading so far. I believe CPython still relies on various bytecodes to run atomically, which you get automatically with the GIL present. So I wonder if hard-to-reproduce concurrency bugs will keep surfacing in the free-threaded CPython for quite some time. I feel like all of this is tragic and Python should have gone to a BEAM-like model some years ago, like as part of the 2 to 3 transition. Instead we get async wreckage and now free threading with its attendant hazards. Plus who knows how many C modules won't be expecting this.
- robertlagrant 2y agoAsync seems fine? What's wrong with it?
- throwaway81523 2y agoWatch this video and maybe you'll understand ;). Warning, NSFW (lots of swearing), use headphones. https://www.youtube.com/watch?v=bzkRVzciAZg https://www.youtube.com/watch?v=bzkRVzciAZg This is also good: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... web search on "colored functions" finds lots of commentary on that article.
- wesselbindt 2y agoI've always found the criticism leveled by the colored functions blog post a bit contrived. Yes, when you replace the words async/await with meaningless concepts I do not care about, it's very annoying to have to arbitrarily mark a function as blue or red. But when you replace the word "aync" with something like "expensive", or "does network calls", it becomes clear that "async/await" makes intrinsic properties about your code (e.g., is it a bad idea to put this call in a loop from a performance perspective) explicit rather than implicit. In short, "await" gives me an extra piece of data about the function, without having to read the body of the function (and the ones it calls, and the ones they call, etc). That's a good thing. There are serious drawbacks to async/await, and the red/blue blog post manages to list none of them. EDIT: all of the above is predicated on the idea that reading code is harder than writing it. If you believe the opposite, then blue/red has a point.
- fuzztester 2y ago[flagged]
- fuzztester 2y agohttps://news.ycombinator.com/item?id=40949956 https://news.ycombinator.com/item?id=40949956
- fuzztester 2y ago[flagged]
- jacob019 2y agoIt's very unpopular to mention Perl, but I did many cool things with it back in the day, and it still holds a special place for me. Perl taught me the power of regex--it's really first class in Perl. I still have some Perl code in production today. But to be fair, it is really easy to write spaghetti in Perl if you don't know what you're doing.
- fuzztester 2y agodon't worry about unpopularity, bro. worry about being true. the rest will take care of itself. if not, you are in the wrong company, forum, or place, and better to work on getting out of there.
- fuzztester 2y ago> But to be fair, it is really easy to write spaghetti in Perl if you don't know what you're doing. bro, you really need to educate yourself some more, if you say things like that. baloney! first of all, perl is already spaghetti. I mean, it has all those curlicues, aka sigils. perl devs love it that way. second of all, that point about being able to write spaghetti is not unique to perl. many people do it in many languages. in fact it is common here on hn to see the statement "you can write fortran in any language".
- saagarjha 2y agoThey’re probably downvoting you because you’ve posted like the laziest trope comment there is. lol Python slow everyone is paid to hide the truth amirite
- eigenvalue 2y agoI personally optimize more for development time and overall productivity in creating and refactoring, adding new features, etc. I'm just so much faster using Python than anything else, it's not even close. There is such an incredible world of great libraries easily available on pip for one thing. Also, I've found that ChatGPT/Claude3.5 are much, much smarter and better at Python than they are at C++ or Rust. I can usually get code that works basically the first or second time with Python, but very rarely can do that using those more performant languages. That's increasingly a huge concern for me as I use these AI tools to speed up my own development efforts very dramatically. Computers are so fast already anyway that the ceiling for optimization of network oriented software that can be done in a mostly async way in Python is already pretty compelling, so then it just comes back again to developer productivity, at least for my purposes.
- indigodaddy 2y agoEver messed about with Claude and php?
- lanstin 2y agoI don't think we are supposed to use HN for humor only posts.
- indigodaddy 2y agoWhy the downvotes? Totally serious question. Jesus Christ HN
- goosejuice 2y agoKind of sounds like you are optimizing for convenience :)
- jodrellblank 2y agohttps://www.servethehome.com/wp-content/uploads/2023/01/Intel-and-AMD-Core-Count-Growth-by-Year-2010-2023.jpg https://www.servethehome.com/wp-content/uploads/2023/01/Inte... AMD EPYC 9754 with 128-cores/256-threads, and EPYC 9734 with 112-cores/224-threads. TomsHardware says they "will compete with Intel's 144-core Sierra Forest chips, which mark the debut of Intel's Efficiency cores (E-cores) in its Xeon data center lineup, and Ampre's 192-core AmpereOne processors". What in 5 years? 10? 20? How long will "1 core should be enough for anyone using Python" stand?
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- d0mine 2y agoNumber crunching code in Python (such as using numpy/pytorch) performs the vast vast majority of its calculations in C/Fortran code under the hood where GIL can be released. Single python process can use multiple CPUs. There is code that may benefit from the free threaded implementation but it is not as often as it might appear and it is not without its own downsides. In general, GIL simplifies multithreaded code. There were no-GIL Python implementations such as Jython, IronPython. They hadn't replaced CPython, Pypy implementation which use GIL i.e., other concerns dominate.
- imachine1980_ 2y agoYes but jython am iron aren't the standard, and I feel the more relevant part is inertia, puppy is design whit lots of concern of compatibility, then being the new standard can totally make difference making both cases not a good comparison.
- phkahler 2y ago>> What in 5 years? 10? 20? How long will "1 core should be enough for anyone using Python" stand? If you're looking for a 32x or 128x performance improvement from python supporting multi-core you should probably rewrite in C, C++, Rust, or Fortran and get that 100x improvement today on a single core. If done properly you can then ALSO get the gain from multiple cores on top of that. Or to put it another way, if performance is critical python is a poor choice.
- Derbasti 2y agoA thought experiment: A piece of code takes 6h to develop in C++, and 1h to run. The same algorithm takes 3h to code in Python, but 6h to run. If I could thread-spam that Python code on my 24 core machine, going Python would make sense. I've certainly been in such situations a few times.
- tho34234234 2y agoIt's not just about "raw-flop performance" though; it affects even basic things like creating data-loaders that run in the background while your main thread is doing some hard ML crunching. Every DL library comes with its own C++ backend that does this for now, but it's annoyingly inflexible. And dealing with GIL is a nightmare if you're dealing with mixed Python code.
- jillesvangurp 2y agoRight now you are right. This is about taking away that argument. There's no technical reason for this to stay true. Other than that the process of fixing this is a lot of work of course. But now that the work has started, it's probably going to progress pretty steadily. It will be interesting to see how this goes over the next few years. My guess is that a lot of lessons were learned from the python 2 to 3 move. This plan seems pretty solid. And of course there's a relatively easy fix for code that can't work without a GIL: just do what people are doing today and just don't fork any threads in python. It's kind of pointless in any case with the GIL in place so not a lot of code actually depends on threads in python. Preventing the forking of threads in the presence of things still requiring the GIL sounds like a good plan. This is a bit of meta data that you could build into packages. This plan is actually proposing keeping track of what packages work without a GIL. So, that should keep people safe enough if dependency tools are updated to make use of this meta data and actively stop people from adding thread unsafe packages when threading is used. So, I have good hopes that this is going to be a much smoother transition than python 2 to 3. The initial phase is probably going to flush out a lot of packages that need fixing. But once those fixes start coming in, it's probably going to be straightforward to move forward.
- paulddraper 2y ago> should not be written IDK what l should and shouldn't be written in, but there are a very large # of proud "pure Python" libraries on GitHub and HN. The ecosystem seems to even prefer them.
- fastasucan 2y agoI never understand this sentiment, that shows up in every topic on python. Who descides why something should or should not be written I Python? Why shouldn't someone who prefers writing in python benefit from using multiple cores?
- phkahler 2y ago>> Who descides why something should or should not be written I Python? Why shouldn't someone who prefers writing in python benefit from using multiple cores? I did use the words "most things". I'm not saying this is a bad development for Python, or that nobody should use it. But if performance is a top priority, Python is the wrong language and always has been. I use Python from time to time, it's fun and easy to put certain kinds of things together quickly. But each time I do a project with it, the first thing I ask myself is "is this going to be fast enough?" If not I'll use something else.
- saurik 2y agoFWIW, I think the concern though is/was that for most of us who aren't doing shared-data multiprocessing this is going to make Python even slower; maybe they figured out how to avoid that?
- eigenvalue 2y agoPretty sure they offset any possible slowdowns by doing heroic optimizations in other parts of CPython. There was even some talk about keeping just those optimizations and leaving the GIL in place, but fortunately they went for the full GILectomy.
- quotemstr 2y agoWhat about the pessimization of single-threaded workloads? I'm still not convinced a completely free-threaded Python is better overall than a multi-interpreter, separate-GIL model with explicit instead of implicit parallelism. Everyone wants parallelism in Python. Removing the GIL isn't the only way to get it.
- Demiurge 2y agoMassive overhead of multiprocessing? How have I not noticed this for tens of years? I use coroutines and multiprocessing all the time, and saturate every core and all the IO, as needed. I use numpy, pandas, xarray, pytorch, etc. How did this terrible GIL overhead completely went unnoticed?
- viraptor 2y ago> I use numpy, pandas, xarray, pytorch, etc. That means your code is using python as glue and you do most of your work completely outside of cPython. That's why you don't see the impact - those libraries drop GIL when you use them, so there's much less overhead.
- quietbritishjim 2y agoThe parent commenter said they're using the multiprocessing module, so it's irrelevant to them whether those modules drop the GIL (except for the fact that they are missing an opportunity to using threading instead). The overhead being referred to, whether significant or not, is that of spawning processes and doing IPC.
- pizza234 2y ago> using simple threads instead of dealing with the massive overhead and complexity and bugs of using something like multiprocessing. Depending on the domain, the reality can be the reverse. Multiprocessing in the web serving domain, as in "spawning separate processes", is actually simpler and less bug-prone, because there is considerably less resource sharing. The considerably higher difficulty of writing, testing and debugging parallel code is evident to anybody who's worked on it. As for the overhead, this again depends on the domain. It's hard to quantify, but generalizing to "massive" is not accurate, especially for app servers with COW support.
- skissane 2y agoJust the other day I was trying to do two things in parallel in Python using threads - and then I switched to multiprocessing - why? I wanted to immediately terminate one thing whenever the other failed. That’s straightforwardly supported with multiprocessing. With threads, it gets a lot more complicated and can involve things with dubious supportability
- lyu07282 2y agoThere is a reason why it's "complicated" in threads, because doing it correctly just IS complicated, and the same reason applies to child processes, you just ignored that reason. That's one example of a footgun in using multiprocessing, people write broken code but they don't know that because it appears to work... until it doesn't (in production on friday night).
- skissane 2y agoI don't agree. A big reason why abruptly terminating threads at an arbitrary point is risky is it can corrupt shared memory. If you aren't using shared memory in a multiprocess solution, that's not an issue. Another big reason is it can lead to resource leaks (e.g. thread gets terminated in a finally clause to close resources and hence the resource doesn't get closed). Again, that's less of an issue for processes, since many resources (file descriptors, network connections) get automatically closed by the OS kernel when the process exits. Abruptly terminating a child process still can potentially cause issues, but there are whole categories of potential issues which exist for abrupt thread termination but not for abrupt process termination.
- wokwokwok 2y ago> there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects This just isn’t true. This does not improve single threaded performance (it’s worse) and concurrent programming is already available. This will make it less annoying to do concurrent processing. It also makes everything slower (arguable where that ends up, currently significantly slower) overall. This way over hyped. At the end of the day this will be a change that (most likely) makes the existing workloads for everyone slightly slower and makes the lives of a few people a bit easier when they implement natively parallel processing like ML easier and better. It’s an incremental win for the ML community, and a meaningless/slight loss for everyone else. At the cost of a great. Deal. Of. Effort. If you’re excited about it because of the hype and don’t really understand it, probably calm down. Mostly likely, at the end of the day, it s a change that is totally meaningless to you, won’t really affect you other than making some libraries you use a bit faster, and others a bit slower. Overall, your standard web application will run a bit slower as a result of it. You probably won’t notice. Your data stack will run a bit faster. That’s nice. That’s it. Over hyped. 100%.
- anwlamp 2y agoYes, good summary. My prediction is that free-threading will be the default at some point because one of the corporations that usurped Python-dev wants it. The rest of us can live with arcane threading bugs and yet another split ecosystem. As I understand it, if a single C-extension opts for the GIL, the GIL will be enabled. Of course the invitation to experiment is meaningless. CPython is run by corporations, many excellent developers have left and people will not have any influence on the outcome.
- Galanwe 2y ago> It's going to be amazing to saturate all the cores on a big machine using simple threads instead of dealing with the massive overhead and complexity and bugs of using something like multiprocessing. I'm saturating 192cpu / 1.5TBram machines with no headache and straightforward multiprocessing. I really don't see what multithreading will bring more. What are these massive overheads / complexity / bugs you're talking about ?
- quietbritishjim 2y agoIf you're worried about performance then much of your CPU time is probably spent in a C extension (e.g. numpy, scipy, opencv, etc.). Those all release the GIL so already allow parallelisation in multiple threads. That even includes many functions in the standard library (e.g. sqlite3, zip/unzip). I've used multiple threads in Python for many years and never needed to break into multiprocessing. But, for sure, nogil will be good for those workloads written in pure Python (though I've personally never been affected by that).
- deleted 2y ago[deleted]
- coldtea 2y ago>using simple threads instead of dealing with the massive overhead and complexity and bugs of using something like multiprocessing I've never heard threading described as "simple", even less so as simpler than multiprocessing. Threads means synchronization issues, shared memory, locking, and other complexities.