5 ms·
I was using gevent in Python about 10 years ago, and from memory, it's roughly similar to goroutines. It's not exactly the same of course, but just like gorouti
by Carpetsmoker 6y ago
I was using gevent in Python about 10 years ago, and from memory, it's roughly similar to goroutines. It's not exactly the same of course, but just like goroutines it's pretty easy to just spawn a few jobs off, wait for them to finish, and get the results.
It's not in the standard library, and there are probably other options too now (not a heavy Python user any more), but Python has had easy parallelism for at least a decade (probably longer).
- eloff 6y agoGevent is like goroutines with GOMAXPROCS=1. Which is to say not nearly as useful. It gives you concurrency without parrallelism, because Python never did shake the GIL. Which goes to show some technical debt will haunt you forever.
- antoinealb 6y agoTo nuance your comment, you can still get some form of parallelism, "just not" thread parallelism in Python. You can still spawn multiple handler process, or have threaded code in a C extension.
- zepto 6y agoIt’s arguable whether either of those are ‘in python’.
- quacker 6y agoThe multiprocessing module in the standard library is absolutely a Python-native way to do parallelism: with Pool(5) as p: print(p.map(f, [1, 2, 3])) This runs f(1), f(2), and f(3) in parallel, using a pool of five processes. https://docs.python.org/3.6/library/multiprocessing.html https://docs.python.org/3.6/library/multiprocessing.html
- goostavos 6y agoWhoa! It never occurred to me that `Pool` could be used with context manager. I've always typed out `.close()` manually like a sucker.
- kbutler 6y agohttps://docs.python.org/2/library/multiprocessing.html https://docs.python.org/2/library/multiprocessing.html Python standard library since 2.6. Pretty much the definition of "in python".
- eloff 6y agoYes, you can use multiple processes to get the parallelism. But that's quite limiting compared with goroutines. Passing data back and forth is hard, and you can petty much forget about shared data structures. Memory usage is also much higher.
- marcosdumay 6y agoThe GIL is not a total ban on thread parallelism. It's a significant obstacle, but not a complete stop. Besides, Go has its own set of problems with parallelism. None of those are best in class.
- throwaway894345 6y agoThis comment makes it sound like Go and Python are pretty much in the same class because neither is perfect. I invite anyone who thinks this way to write considerable (shared-memory) parallel code in both languages and see which they prefer. As for Go's "set of problems with parallelism", they're pretty much just that sharing memory is hard to do correctly without giving up performance. No languages do this well; Rust and Haskell make it appear easier by making single-threaded code more difficult to write--requiring you to adhere to invariants like functional purity or borrowing. If you're writing Python, you very likely have values that are incompatible with these invariants (you want to onboard new developers quickly and you want your developers to write code quickly and you're willing to trade off on correctness to do so). Go is absolutely best-in-class if you have typical Python values.
- marcosdumay 6y ago> Go is absolutely best-in-class if you have typical Python values. "Best in class" is not a relative term. Neither Go nor Python are appropriate choices for highly parallel intercomunicating code. Yes, Python is more limited than Go here, but hardly makes a difference when you avoid it. Haskell and Rust do make it easier, by forcing developers to organize their code in a completely different way. Erlang does the same, with a different kind of organization. None of those languages are more difficult to program in, but yes, they are hard to learn.
- jatone 6y ago> Neither Go nor Python are appropriate choices for highly parallel intercomunicating code. thats an extraordinary claim which needs evidence.
- quacker 6y ago> Gevent is like goroutines with GOMAXPROCS=1. On its own, yes. For webapps, you can easily combine it with a multi-process WSGI server (like gunicorn or similar).
- throwaway894345 6y agoRunning four processes with GOMAXPROCS=1 is strictly worse than 1 process with GOMAXPROCS=4. The difference is coarse-vs-fine grained parallelism. Notably, a WSGI+gevent system doesn't allow you to do parallelism within a request, not to mention configuring these WSGI implementations (especially for production) is a bunch of extra headache for which there is no analogy in Go.
- sandGorgon 6y agoFunny you mention the GIL - this is being done as part of PEP 554 which was slated to release in python 3.9 (in alpha). And it's showing great promise ...while it could be delayed. https://www.mail-archive.com/python-dev@python.org/msg108063.html https://www.mail-archive.com/python-dev@python.org/msg108063... 2021 is probable going to be the "Gone Gil" moment !
- amelius 6y agoWhy don't they use/fork the GoLang runtime?
- YawningAngel 6y agoThere are a few reasons: 1. This would definitely break the CPython API, which is not an option for mainstream Python. 2. The Golang runtime isn't really well-understood as a backend for languages that aren't Golang, although I acknowledge that there's no reason in principle why you couldn't compile $LANGUAGE to Golang.
- meowface 6y agoI still use and greatly prefer gevent to this day. They are indeed quite similar to goroutines. Asyncio is a different model, and more irritating for me to work with. (I'm sure this isn't the case for everyone. I'm just more productive with gevent, personally.) Performance is pretty close for both. I was disappointed to not see Python get an official green thread implementation. The counter-argument I commonly see cited is https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html. I personally don't find it to be a very convincing argument.
- sandGorgon 6y agoWhat's missing in asyncio when compared to gevent ? The coroutine and queue model is the same right ? Cool thing in 3.8: Running python -m asyncio launches a natively async REPL.
- meowface 6y agoThey're just completely different models. gevent is green threads, asyncio is explicit coroutines. In gevent, you don't use an "async" or "await" syntax. You just spawn greenlets (green threads), which run in the background, and if you want, you can have them block until they return a result. And they're efficient, so you can spawn thousands without a problem.
- quacker 6y agoI always considered "green threads" and "coroutines" to be the same thing? Are they not?
- deleted 6y ago[deleted]
- infinite8s 6y agoThey are both built on the same technology in the CPython runtime.
- 6y ago