4 ms·

That raises a few questions : 1- will this takes us closer to a golang like way of working in parallel ( since that means no GIL) ? 2- Why did dropbox / guido

by bsaul 12y ago

That raises a few questions : 1- will this takes us closer to a golang like way of working in parallel ( since that means no GIL) ? 2- Why did dropbox / guido didn't join forces with pypy, since they already had proved that their approach was very efficient performance wise ? Is it because pypy doesn't deal with native C code at all, whereas the python LLVM approach they chose may let the two work better ? 3- does anyone else feel like the python world is a complete mess right now ? EDIT : didn't want to sound harsh, on the contrary. Seing so many smart people work on separate path on great projects for so long feels like some great energy is going to end up a waste of time.

meowface 12y ago
>1- will this takes us closer to a golang like way of working in parallel ( since that means no GIL) ? I doubt it. The asyncio module was an attempt at this but the API isn't that great and it also isn't closely integrated with the language like goroutines are in Golang. This is still a great improvement though, as is PyPy in general. Basically, currently existing programs should run faster, and people will use the multiprocessing module less and the threading module more. It may change the way lower level parallel code is written, and library maintainers may be able to simplify some things. >2- Why did dropbox / guido didn't join forces with pypy, since they already had proved that their approach was very efficient performance wise ? Is it because pypy doesn't deal with native C code at all, whereas the python LLVM approach they chose may let the two work better? Probably a mix of "NIH" and also the belief that Dropbox has some of the best Python devs (and BDFLs) out there, so surely they can create the best solution. We'll see in the next few years if it plays out for them. >3- does anyone else feel like the python world is a complete mess right now ? Not that much. It's a big mess if you're trying to maintain a large open source project and need to keep retaining compatibility from 2.6 to 3.4+, but as a regular developer it's not too terrible. Personally I am still going to stick with 2.7 for personal projects for years to come, though.
- servercobra 12y ago
  For 3, even that isn't terrible in a lot of cases. All of OpenStack (AFAIK) maintains py26, py27 and py33 support (and one of the projects I work on aims for pypy as well, but isn't gating on it yet).
  - meowface 12y ago
    It's not that terrible, but it can be annoying. And arguably worst of all, it can make code a lot uglier than it needs to be.
ak217 12y ago
1. Yes, thanks to Armin's great work, this improves language support for parallelism. It doesn't improve support for concurrency patterns (like coroutines and channels). Python now has concurrent.futures, yield from, and the (overly complicated) asyncio library, which provide a toolkit for concurrent programming patterns, but it's not really beautiful nor a joy to use. That's part 2 of Armin's proposal, and he's asking for donations to fund it (see the link above). 2. Until recently PyPy was still regarded as an experimental side project, and had memory use limitations that prevented it from being a drop-in CPython replacement. CPython is a simpler codebase than PyPy and supports more targets and environments. PyPy does deal with native C code via cffi, but there is a lot of code that uses CPython-specific C bindings that will need to be updated to run on PyPy. 3. I think the Python world is doing great. The Python 2 to 3 and CPython to PyPy migrations are largely orthogonal, and will coalesce to "PyPy 3000 with STM" soon enough. Once that happens, I think that will serve as a really powerful foundation for future improvements. (Writing code that runs on both 2.7 and 3.3+ is not that hard, by the way.)
MetaCosm 12y ago
1. Well -- unless I misunderstand, they use STM to emulate a GIL (while actually letting stuff be done concurrently). Which means that it will still be a bit of a hodgepodge messy disaster. 2. I can't speak for Dropbox, but I worked in a shop that bet their future on Python -- and we spent a good bit of effort looking at PyPy ... it was a nightmare if anything went wrong -- a horrific impossible to debug mess. 3. Yep, I added Python to my "won't work on" list of languages (with Cobol, Progress and MUMPS).
- fijal 12y ago
  What sort of problems did you run into? Did you submit bug reports? I don't think finding a bug in say V8 JIT will be easy to debug for example (or any advanced VM).

by bsaul 12y ago

meowface 12y ago

>1- will this takes us closer to a golang like way of working in parallel ( since that means no GIL) ? I doubt it. The asyncio module was an attempt at this but the API isn't that great and it also isn't closely integrated with the language like goroutines are in Golang. This is still a great improvement though, as is PyPy in general. Basically, currently existing programs should run faster, and people will use the multiprocessing module less and the threading module more. It may change the way lower level parallel code is written, and library maintainers may be able to simplify some things. >2- Why did dropbox / guido didn't join forces with pypy, since they already had proved that their approach was very efficient performance wise ? Is it because pypy doesn't deal with native C code at all, whereas the python LLVM approach they chose may let the two work better? Probably a mix of "NIH" and also the belief that Dropbox has some of the best Python devs (and BDFLs) out there, so surely they can create the best solution. We'll see in the next few years if it plays out for them. >3- does anyone else feel like the python world is a complete mess right now ? Not that much. It's a big mess if you're trying to maintain a large open source project and need to keep retaining compatibility from 2.6 to 3.4+, but as a regular developer it's not too terrible. Personally I am still going to stick with 2.7 for personal projects for years to come, though.

servercobra 12y ago
For 3, even that isn't terrible in a lot of cases. All of OpenStack (AFAIK) maintains py26, py27 and py33 support (and one of the projects I work on aims for pypy as well, but isn't gating on it yet).
- meowface 12y ago
  It's not that terrible, but it can be annoying. And arguably worst of all, it can make code a lot uglier than it needs to be.

ak217 12y ago

1. Yes, thanks to Armin's great work, this improves language support for parallelism. It doesn't improve support for concurrency patterns (like coroutines and channels). Python now has concurrent.futures, yield from, and the (overly complicated) asyncio library, which provide a toolkit for concurrent programming patterns, but it's not really beautiful nor a joy to use. That's part 2 of Armin's proposal, and he's asking for donations to fund it (see the link above). 2. Until recently PyPy was still regarded as an experimental side project, and had memory use limitations that prevented it from being a drop-in CPython replacement. CPython is a simpler codebase than PyPy and supports more targets and environments. PyPy does deal with native C code via cffi, but there is a lot of code that uses CPython-specific C bindings that will need to be updated to run on PyPy. 3. I think the Python world is doing great. The Python 2 to 3 and CPython to PyPy migrations are largely orthogonal, and will coalesce to "PyPy 3000 with STM" soon enough. Once that happens, I think that will serve as a really powerful foundation for future improvements. (Writing code that runs on both 2.7 and 3.3+ is not that hard, by the way.)

MetaCosm 12y ago

1. Well -- unless I misunderstand, they use STM to emulate a GIL (while actually letting stuff be done concurrently). Which means that it will still be a bit of a hodgepodge messy disaster. 2. I can't speak for Dropbox, but I worked in a shop that bet their future on Python -- and we spent a good bit of effort looking at PyPy ... it was a nightmare if anything went wrong -- a horrific impossible to debug mess. 3. Yep, I added Python to my "won't work on" list of languages (with Cobol, Progress and MUMPS).

fijal 12y ago
What sort of problems did you run into? Did you submit bug reports? I don't think finding a bug in say V8 JIT will be easy to debug for example (or any advanced VM).