5 ms·

The entire Python ecosystem is like this. Different conflicting versions of the language, different package managers broken in different ways, etc. It’s so bad

by woah 6y ago

The entire Python ecosystem is like this. Different conflicting versions of the language, different package managers broken in different ways, etc. It’s so bad that the core user base of data scientists usually just use machines that have all their libraries pre installed. Contrast this with a language like Rust or JS, where you run one command once to install everything

ubercore 6y ago
I don't know Rust well enough to comment, but doing a lot of development in both JS and Python, JS is no cakewalk either. I'd say I deal with at least as many packaging/versioning issues on JS as I do on Python.
zo1 6y ago
The JS ecosystem has just as many "issues" if you look at it objectively. Don't mean to be funny or lazy, but one need only look at this SO answer to see how weird/fragmented the JS package system is (the answer has a Table of Contents). https://stackoverflow.com/a/39825582/1140382 https://stackoverflow.com/a/39825582/1140382
- woah 6y ago
  That's not the JS package ecosystem. Those are all various techniques from 5 years ago for packaging frontend code to save download bandwidth.
meowface 6y ago
Another issue is the two mutually incompatible ecosystems of async vs. non-async Python libraries and frameworks. You now have these entire mirror universes of code, with "[X]-async" versions of existing synchronous libraries, and new async frameworks which require using these libraries. The years of work that've gone into non-async libraries are now basically all for naught, with lots of people reinventing the wheel for every possible thing that interacts with a network. (I'm also not a fan of async/await in general.) I'm a big Python fan; it was my first language and still by far my most-used and the one I know best. But the "developer experience" is undoubtedly a giant mess compared to something like Go or Rust. (Rust also appears to be having some growing pains related to async, but everything else seems way more solid.) Despite Python's catchphrase "there should be one, and preferably only one, way to do it", Go really embodies this across every dimension. One way to install the runtime, one way to format, one way to package, one way to do concurrency, one way to test, one way to write most things. And every version is almost completely backwards compatible. It's kind of crazy that a language that (justifiably) prides itself in its programming simplicity also makes you deal with tangled messes like https://xkcd.com/1987/ https://xkcd.com/1987/ My one solace is that some all-in-one third-party tooling is finally close to the level of convenience you get with Rust. pyenv and poetry save a ton of headache when it comes to managing Python versions and packages, and "just work" in a pretty similar way as rustup and cargo.
- viraptor 6y ago
  > The years of work that've gone into non-async libraries are now basically all for naught, That's fud. Non-async code lives in, is maintained, and is not affected by the async alternatives poping up. We had libraries based on twisted for over a decade, which were basically in the same position, but you're not mentioning those? > It's kind of crazy that a language that (justifiably) prides itself in its programming simplicity also makes you deal with tangled messes That comic is satire with many paths which don't really exist. Reality is much simpler is you're explicit about which version you're using and use a single source rather than OS, homebrew, Anaconda and upstream release at the same time. With one version manager and virtual environment per project everything works almost exactly like in Rust.
  - meowface 6y ago
    You're right, "all for naught" was going too far and I apologize for the hyperbole, but it's kind of like a blockchain fork: most new projects are gradually moving to the new async side of things, and this new mirror universe is being built at an accelerating pace while the previous fork (synchronous libraries) is decelerating. In 5, 10, 15 years, a lot of code that might have otherwise still had some utility may no longer be practical. A lot of such code might have become obsolete after that long anyway, but some of it might be used far less than it could've been solely because it's outside of the future async status quo. There are some awesome Perl projects out there, and their work definitely wasn't for naught, but it's hard to justify using them for anything serious in 2020, kind of like how it might end up being hard to justify using synchronous Python in 2025 or 2030. (Not in the sense of running a standalone application, in which case who cares what it's written with, but in terms of something that may require some kind of continuous development or interfacing with other code.) >We had libraries based on twisted for over a decade, which were basically in the same position, but you're not mentioning those? My understanding is Twisted doesn't suffer from the so-called "what color is your function problem": http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... With Twisted, you need to swap out certain network calls with Twisted's own APIs to use its full feature set, but it has an easy way to defer existing synchronous/blocking code: https://twistedmatrix.com/documents/12.2.0/core/howto/gendefer.html#auto5 https://twistedmatrix.com/documents/12.2.0/core/howto/gendef... There's no way to do something like this with async/await, besides some hacky third-party attempts, as far as I know. But, yes, I'm not a huge fan of Twisted's model, either. I was thinking more along the lines of a green thread model like gevent, which, although controversial, doesn't require you to change a single line of code beyond a monkey patch line at the top of the file. I'm not really swayed by Twisted's creator's anti-green thread manifesto (https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html). And any language with a baked-in green thread model like Go also avoids this problem. Synchronous code can be run asynchronously, or synchronously, with no red/blue barrier. >That comic is satire with many paths which don't really exist. Yes, of course it's hyperbole, but it's not that far off, in my opinion. Now that I use pyenv for everything I no longer run into any issues like these, but before, it could sometimes become a nightmare. I love tools like pyenv and poetry and they make Python way easier to deal with (it's still my favorite language, for sure), but part of the point of this thread is that it would be nice if universal tools like these were baked into the actual release like how Rust's and Go's tools are baked into theirs. Tons of people still aren't aware that things like pyenv or poetry or Anaconda exist, or that they should consider trying them. Or they want to try them but are dissuaded due to work policy about changing things or using new third-party tools, or even any kind of third-party tools. Someone else in this thread brought this up here: https://news.ycombinator.com/item?id=25255536 https://news.ycombinator.com/item?id=25255536

by woah 6y ago

ubercore 6y ago

I don't know Rust well enough to comment, but doing a lot of development in both JS and Python, JS is no cakewalk either. I'd say I deal with at least as many packaging/versioning issues on JS as I do on Python.

zo1 6y ago

The JS ecosystem has just as many "issues" if you look at it objectively. Don't mean to be funny or lazy, but one need only look at this SO answer to see how weird/fragmented the JS package system is (the answer has a Table of Contents). https://stackoverflow.com/a/39825582/1140382 https://stackoverflow.com/a/39825582/1140382

woah 6y ago
That's not the JS package ecosystem. Those are all various techniques from 5 years ago for packaging frontend code to save download bandwidth.

meowface 6y ago

Another issue is the two mutually incompatible ecosystems of async vs. non-async Python libraries and frameworks. You now have these entire mirror universes of code, with "[X]-async" versions of existing synchronous libraries, and new async frameworks which require using these libraries. The years of work that've gone into non-async libraries are now basically all for naught, with lots of people reinventing the wheel for every possible thing that interacts with a network. (I'm also not a fan of async/await in general.) I'm a big Python fan; it was my first language and still by far my most-used and the one I know best. But the "developer experience" is undoubtedly a giant mess compared to something like Go or Rust. (Rust also appears to be having some growing pains related to async, but everything else seems way more solid.) Despite Python's catchphrase "there should be one, and preferably only one, way to do it", Go really embodies this across every dimension. One way to install the runtime, one way to format, one way to package, one way to do concurrency, one way to test, one way to write most things. And every version is almost completely backwards compatible. It's kind of crazy that a language that (justifiably) prides itself in its programming simplicity also makes you deal with tangled messes like https://xkcd.com/1987/ https://xkcd.com/1987/ My one solace is that some all-in-one third-party tooling is finally close to the level of convenience you get with Rust. pyenv and poetry save a ton of headache when it comes to managing Python versions and packages, and "just work" in a pretty similar way as rustup and cargo.

viraptor 6y ago
> The years of work that've gone into non-async libraries are now basically all for naught, That's fud. Non-async code lives in, is maintained, and is not affected by the async alternatives poping up. We had libraries based on twisted for over a decade, which were basically in the same position, but you're not mentioning those? > It's kind of crazy that a language that (justifiably) prides itself in its programming simplicity also makes you deal with tangled messes That comic is satire with many paths which don't really exist. Reality is much simpler is you're explicit about which version you're using and use a single source rather than OS, homebrew, Anaconda and upstream release at the same time. With one version manager and virtual environment per project everything works almost exactly like in Rust.
- meowface 6y ago
  You're right, "all for naught" was going too far and I apologize for the hyperbole, but it's kind of like a blockchain fork: most new projects are gradually moving to the new async side of things, and this new mirror universe is being built at an accelerating pace while the previous fork (synchronous libraries) is decelerating. In 5, 10, 15 years, a lot of code that might have otherwise still had some utility may no longer be practical. A lot of such code might have become obsolete after that long anyway, but some of it might be used far less than it could've been solely because it's outside of the future async status quo. There are some awesome Perl projects out there, and their work definitely wasn't for naught, but it's hard to justify using them for anything serious in 2020, kind of like how it might end up being hard to justify using synchronous Python in 2025 or 2030. (Not in the sense of running a standalone application, in which case who cares what it's written with, but in terms of something that may require some kind of continuous development or interfacing with other code.) >We had libraries based on twisted for over a decade, which were basically in the same position, but you're not mentioning those? My understanding is Twisted doesn't suffer from the so-called "what color is your function problem": http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... With Twisted, you need to swap out certain network calls with Twisted's own APIs to use its full feature set, but it has an easy way to defer existing synchronous/blocking code: https://twistedmatrix.com/documents/12.2.0/core/howto/gendefer.html#auto5 https://twistedmatrix.com/documents/12.2.0/core/howto/gendef... There's no way to do something like this with async/await, besides some hacky third-party attempts, as far as I know. But, yes, I'm not a huge fan of Twisted's model, either. I was thinking more along the lines of a green thread model like gevent, which, although controversial, doesn't require you to change a single line of code beyond a monkey patch line at the top of the file. I'm not really swayed by Twisted's creator's anti-green thread manifesto (https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html). And any language with a baked-in green thread model like Go also avoids this problem. Synchronous code can be run asynchronously, or synchronously, with no red/blue barrier. >That comic is satire with many paths which don't really exist. Yes, of course it's hyperbole, but it's not that far off, in my opinion. Now that I use pyenv for everything I no longer run into any issues like these, but before, it could sometimes become a nightmare. I love tools like pyenv and poetry and they make Python way easier to deal with (it's still my favorite language, for sure), but part of the point of this thread is that it would be nice if universal tools like these were baked into the actual release like how Rust's and Go's tools are baked into theirs. Tons of people still aren't aware that things like pyenv or poetry or Anaconda exist, or that they should consider trying them. Or they want to try them but are dissuaded due to work policy about changing things or using new third-party tools, or even any kind of third-party tools. Someone else in this thread brought this up here: https://news.ycombinator.com/item?id=25255536 https://news.ycombinator.com/item?id=25255536