4 ms·

This post is prompted by the survey of Python users, their feedback, and a current thread [0] on the Python forms discussing a way forward. I read the thread th

This post is prompted by the survey of Python users, their feedback, and a current thread [0] on the Python forms discussing a way forward. I read the thread the other day, it's long, and there are a lot of opinions. Towards the end of this post there is an interesting observation: > Discourse, the platform that the discussion was held on, shows the number of times a link was clicked. Granted, this count might not be always accurate, but if we assume it is, the link to the results summary was clicked only 14 times (as of 2023-01-14 21:20 UTC). The discussion has 28 participants and 2.2k views. If we believe the link click counter, half of the discussion participants did not even bother reading what the people think. Along with that, my concern is that there are too my cooks in the kitchen. A packaging system is never going to cater for more than 90% of use cases, but there will likely be a higher than average representation from people who would sit in the 10%. There is a danger of them never being able to agree on a solution that solves all problems, and then never launch anything. It would be far better to get something out, have a stake in the ground, and build from there. It also desperately needs to be "Core Python", my understanding is that the PyPA (Python Package Authority) is somewhat disconnected from the core Python development. Ultimately they need one system that is front and centre, that is the "officially endorsed" tool, and very clear messaging on that. 0: https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420 https://discuss.python.org/t/python-packaging-strategy-discu...

forkerenok 4y ago
Thanks, this is an interesting nugget. Did you intend to append a link reference for [0]?
- samwillis 4y ago
  Yes, edited. Thanks.
mkoubaa 4y ago
I always like to say that's barring divine inspiration, it's impossible to make the right choice. It's far more possible to make a choice and then work hard to make it right
carapace 4y ago
I think you're right. At risk of making a "lowbrow dismissal" the very length of TFA is a clear symptom of the problem: Design-by-committee (The PyPA (Python Package Authority) reminds me of the movie Brazil. Don't get me started.) - - - - Someone should do for packaging what pathlib did for file I/O: make a Pythonic model/API for it. Then deployment becomes simple, scriptable, testable, repeatable, etc...
- woodruffw 4y ago
  This might be a fun thought to entertain, but the PyPA doesn't really form design committees (at least, I haven't been on one). You can see exactly how Python packaging standards are made: they're done with PEPs[1], exactly the same as with every other Python standardization effort. Indeed, most packaging PEPs start out exactly the way you've laid out: a tool or service writes a Pythonic API or model, and it gets standardized so that other tools can rely on it. TFA's problems (which are real ones!) stem mostly from the error before there were serious standardization efforts among Python packaging tools. [1]: https://peps.python.org/topic/packaging/ https://peps.python.org/topic/packaging/
  - carapace 4y ago
    > Don't get me started.
noisy_boy 4y ago
> Ultimately they need one system that is front and centre, that is the "officially endorsed" tool, and very clear messaging on that. I think this is important; otherwise there is a risk behind switching to a new packaging tool and then it being subject to neglect/lack of resources - that makes people averse to switching to it. That is why virtualenv/pip is the lowest common denominator - everyone knows that worst case, those will always continue to work. The "official" tool needs to inspire that sort of confidence.
- nmstoker 4y ago
  Yes, there was some questionable comms around pipenv being "officially recommdnded" which upon closer inspection seemed to have come from the people who wrote it and not really been that official so far as I could see! That seems to have been walked back after a while but not before it gained traction and randoms bring up that it's official even now.
  - KyeRussell 4y ago
    I really hate to be this “inside baseball” and negative about it all, but I think that the Kenneth Reitz factor is…unique (within the Python community), and uniquely dangerous in these situations. I am so deeply fed up with the Python packaging experience. At this point, both my eyes are set on the fastest way to some sort of standardisation and improvement. Its because of this that I want to clearly acknowledge that IMO over the last 5 years we were thrown off course in no small part by one person’s lust for personal notoriety when the community clearly needed to bring their heads together on a solution. Pipenv was touted as the be all end all way too quickly, as you said not necessarily by ‘official sources’, and those shaky shaky years left everyone a bit battered. When I did my every-12-months checkin to see how Python dependency management was going, I sure enough saw that Pipenv is ‘under’ PyPA now. I then learned that a PyPA endorsement or association. doesn’t mean a HUGE amount for the reasons that you and others have already noted. Sure enough, PyPA’s own packaging documentation is hesitant to strongly prefer a front end tool let alone any other part of the stack. A significantly improved Python dependency management story could very well include Pipenv on the front end for all I care. But not acknowledging the elephant in the room risks repeating past mistakes. Lest we end up with more of the celebrity developer culture we see in communities like JS, but without the seemingly limitless effort and resources. None of this is to understate the technical challenges, especially taking into account pulling together the current fragmented ecosystem.
marsten 4y ago
> there will likely be a higher than average representation from people who would sit in the 10% And also an under-representation of "average users". Python the language has always benefitted from its relative simplicity, which I attribute to people like GVR saying "no" to specialized features that accrue on languages like barnacles on a ship (looking at you C++). With newer languages we see core design teams being much more opinionated about tooling for build and dependendency/packaging management. This is no doubt in response to the dumpster fires of C++ and Python, where "the ecosystem will sort itself out" clearly hasn't worked.
- pjmlp 4y ago
  I know Python since version 1.6, its simplicity is deceptive, the language is as rich as C++, beginners only think otherwise because the language looks simple on the surface. When one starts exploring the standard library, everything that is exposed in meta-programming and runtime internals, or the usual language changes even across minor versions, the picture changes dramatically. Once upon a time I always devoured the release notes for each release, eventually I lost track of changes.
  - Yoric 4y ago
    A few years ago, I worked on a (company-strategic) experimental project that used Python. Apparently, it only worked on some specific versions of Python, due to changes in the internal order of sets, a fact that was of course not documented or checked in the tool. Not the reason for which the project failed, but the weeks lost trying to replicate results between teams certainly didn't help. Python is very readable, but I agree that the stdlib is anything but simple.

forkerenok 4y ago

Thanks, this is an interesting nugget. Did you intend to append a link reference for [0]?

samwillis 4y ago
Yes, edited. Thanks.

mkoubaa 4y ago

I always like to say that's barring divine inspiration, it's impossible to make the right choice. It's far more possible to make a choice and then work hard to make it right

carapace 4y ago

I think you're right. At risk of making a "lowbrow dismissal" the very length of TFA is a clear symptom of the problem: Design-by-committee (The PyPA (Python Package Authority) reminds me of the movie Brazil. Don't get me started.) - - - - Someone should do for packaging what pathlib did for file I/O: make a Pythonic model/API for it. Then deployment becomes simple, scriptable, testable, repeatable, etc...

woodruffw 4y ago
This might be a fun thought to entertain, but the PyPA doesn't really form design committees (at least, I haven't been on one). You can see exactly how Python packaging standards are made: they're done with PEPs[1], exactly the same as with every other Python standardization effort. Indeed, most packaging PEPs start out exactly the way you've laid out: a tool or service writes a Pythonic API or model, and it gets standardized so that other tools can rely on it. TFA's problems (which are real ones!) stem mostly from the error before there were serious standardization efforts among Python packaging tools. [1]: https://peps.python.org/topic/packaging/ https://peps.python.org/topic/packaging/
- carapace 4y ago
  > Don't get me started.

noisy_boy 4y ago

> Ultimately they need one system that is front and centre, that is the "officially endorsed" tool, and very clear messaging on that. I think this is important; otherwise there is a risk behind switching to a new packaging tool and then it being subject to neglect/lack of resources - that makes people averse to switching to it. That is why virtualenv/pip is the lowest common denominator - everyone knows that worst case, those will always continue to work. The "official" tool needs to inspire that sort of confidence.

nmstoker 4y ago
Yes, there was some questionable comms around pipenv being "officially recommdnded" which upon closer inspection seemed to have come from the people who wrote it and not really been that official so far as I could see! That seems to have been walked back after a while but not before it gained traction and randoms bring up that it's official even now.
- KyeRussell 4y ago
  I really hate to be this “inside baseball” and negative about it all, but I think that the Kenneth Reitz factor is…unique (within the Python community), and uniquely dangerous in these situations. I am so deeply fed up with the Python packaging experience. At this point, both my eyes are set on the fastest way to some sort of standardisation and improvement. Its because of this that I want to clearly acknowledge that IMO over the last 5 years we were thrown off course in no small part by one person’s lust for personal notoriety when the community clearly needed to bring their heads together on a solution. Pipenv was touted as the be all end all way too quickly, as you said not necessarily by ‘official sources’, and those shaky shaky years left everyone a bit battered. When I did my every-12-months checkin to see how Python dependency management was going, I sure enough saw that Pipenv is ‘under’ PyPA now. I then learned that a PyPA endorsement or association. doesn’t mean a HUGE amount for the reasons that you and others have already noted. Sure enough, PyPA’s own packaging documentation is hesitant to strongly prefer a front end tool let alone any other part of the stack. A significantly improved Python dependency management story could very well include Pipenv on the front end for all I care. But not acknowledging the elephant in the room risks repeating past mistakes. Lest we end up with more of the celebrity developer culture we see in communities like JS, but without the seemingly limitless effort and resources. None of this is to understate the technical challenges, especially taking into account pulling together the current fragmented ecosystem.

marsten 4y ago

> there will likely be a higher than average representation from people who would sit in the 10% And also an under-representation of "average users". Python the language has always benefitted from its relative simplicity, which I attribute to people like GVR saying "no" to specialized features that accrue on languages like barnacles on a ship (looking at you C++). With newer languages we see core design teams being much more opinionated about tooling for build and dependendency/packaging management. This is no doubt in response to the dumpster fires of C++ and Python, where "the ecosystem will sort itself out" clearly hasn't worked.

pjmlp 4y ago
I know Python since version 1.6, its simplicity is deceptive, the language is as rich as C++, beginners only think otherwise because the language looks simple on the surface. When one starts exploring the standard library, everything that is exposed in meta-programming and runtime internals, or the usual language changes even across minor versions, the picture changes dramatically. Once upon a time I always devoured the release notes for each release, eventually I lost track of changes.
- Yoric 4y ago
  A few years ago, I worked on a (company-strategic) experimental project that used Python. Apparently, it only worked on some specific versions of Python, due to changes in the internal order of sets, a fact that was of course not documented or checked in the tool. Not the reason for which the project failed, but the weeks lost trying to replicate results between teams certainly didn't help. Python is very readable, but I agree that the stdlib is anything but simple.