6 ms·
Gevent is like goroutines with GOMAXPROCS=1. Which is to say not nearly as useful. It gives you concurrency without parrallelism, because Python never did shake
by eloff 7y ago
Gevent is like goroutines with GOMAXPROCS=1. Which is to say not nearly as useful. It gives you concurrency without parrallelism, because Python never did shake the GIL. Which goes to show some technical debt will haunt you forever.
- antoinealb 7y agoTo nuance your comment, you can still get some form of parallelism, "just not" thread parallelism in Python. You can still spawn multiple handler process, or have threaded code in a C extension.
- zepto 7y agoIt’s arguable whether either of those are ‘in python’.
- quacker 7y agoThe multiprocessing module in the standard library is absolutely a Python-native way to do parallelism: with Pool(5) as p: print(p.map(f, [1, 2, 3])) This runs f(1), f(2), and f(3) in parallel, using a pool of five processes. https://docs.python.org/3.6/library/multiprocessing.html https://docs.python.org/3.6/library/multiprocessing.html
- goostavos 7y agoWhoa! It never occurred to me that `Pool` could be used with context manager. I've always typed out `.close()` manually like a sucker.
- kbutler 7y agohttps://docs.python.org/2/library/multiprocessing.html https://docs.python.org/2/library/multiprocessing.html Python standard library since 2.6. Pretty much the definition of "in python".
- eloff 7y agoYes, you can use multiple processes to get the parallelism. But that's quite limiting compared with goroutines. Passing data back and forth is hard, and you can petty much forget about shared data structures. Memory usage is also much higher.
- marcosdumay 7y agoThe GIL is not a total ban on thread parallelism. It's a significant obstacle, but not a complete stop. Besides, Go has its own set of problems with parallelism. None of those are best in class.
- throwaway894345 7y agoThis comment makes it sound like Go and Python are pretty much in the same class because neither is perfect. I invite anyone who thinks this way to write considerable (shared-memory) parallel code in both languages and see which they prefer. As for Go's "set of problems with parallelism", they're pretty much just that sharing memory is hard to do correctly without giving up performance. No languages do this well; Rust and Haskell make it appear easier by making single-threaded code more difficult to write--requiring you to adhere to invariants like functional purity or borrowing. If you're writing Python, you very likely have values that are incompatible with these invariants (you want to onboard new developers quickly and you want your developers to write code quickly and you're willing to trade off on correctness to do so). Go is absolutely best-in-class if you have typical Python values.
- marcosdumay 7y ago> Go is absolutely best-in-class if you have typical Python values. "Best in class" is not a relative term. Neither Go nor Python are appropriate choices for highly parallel intercomunicating code. Yes, Python is more limited than Go here, but hardly makes a difference when you avoid it. Haskell and Rust do make it easier, by forcing developers to organize their code in a completely different way. Erlang does the same, with a different kind of organization. None of those languages are more difficult to program in, but yes, they are hard to learn.
- jatone 7y ago> Neither Go nor Python are appropriate choices for highly parallel intercomunicating code. thats an extraordinary claim which needs evidence.
- quacker 7y ago> Gevent is like goroutines with GOMAXPROCS=1. On its own, yes. For webapps, you can easily combine it with a multi-process WSGI server (like gunicorn or similar).
- throwaway894345 7y agoRunning four processes with GOMAXPROCS=1 is strictly worse than 1 process with GOMAXPROCS=4. The difference is coarse-vs-fine grained parallelism. Notably, a WSGI+gevent system doesn't allow you to do parallelism within a request, not to mention configuring these WSGI implementations (especially for production) is a bunch of extra headache for which there is no analogy in Go.
- sandGorgon 7y agoFunny you mention the GIL - this is being done as part of PEP 554 which was slated to release in python 3.9 (in alpha). And it's showing great promise ...while it could be delayed. https://www.mail-archive.com/python-dev@python.org/msg108063.html https://www.mail-archive.com/python-dev@python.org/msg108063... 2021 is probable going to be the "Gone Gil" moment !
- amelius 7y agoWhy don't they use/fork the GoLang runtime?
- YawningAngel 7y agoThere are a few reasons: 1. This would definitely break the CPython API, which is not an option for mainstream Python. 2. The Golang runtime isn't really well-understood as a backend for languages that aren't Golang, although I acknowledge that there's no reason in principle why you couldn't compile $LANGUAGE to Golang.