4 ms·

> what makes this so difficult to solve in Python? Python creates the perfect storm for package management hell: - Most the valuable libraries are natively co

> what makes this so difficult to solve in Python? Python creates the perfect storm for package management hell: - Most the valuable libraries are natively compiled (so you get all the fun of distributing binaries for every platform without any of the traditional benefits of native compilation) - The dynamic nature makes it challenging to understand the non-local impacts of changes without a full integration test suite (library developers break each other all the time without realizing it, semantic versioning is a farce) - Too many fractured packaging solutions, not a single one well designed. And they all conflict. - A bifurcated culture of interactive use vs production code - while they both ostensibly use the same language, they have wildly different sub-cultures and best practices. - Churn: a culture that largely disavows strong backwards compatibility guarantees, in favor of the "move fast and break things" approach. (Consequence: you have to move fast too just to keep up with all the breakage) - A culture that values ease of use above simplicity of implementation. Python developers would rather save 1 line of code in the moment, even if it pushes the complexity off to another part of the system. The quite obvious consequence is an ever-growing backlog of complexity. Some of the issues are technical. But I'd argue that the final bullet is why all of the above problems are getting worse, not better.

aaroninsf 2y ago
I agree with all of these and it makes me wonder as I do from time to time, has anyone managed to make a viable P#, a clean break which retains most of what most people love about the language and environment; and cheerfully asserts new and immutable change in things like <the technical parts of the above>. When I have looked into this it seems people can't help but improve one-more-thing or one-other-thing and end up just enjoying vaguely-pythonic language design.
- aaroninsf 2y ago
  Googling P# led me to this delight which is 100% unrelated: https://couragetotremble.blog/2007/08/09/p-language/ https://couragetotremble.blog/2007/08/09/p-language/
- graemep 2y ago
  IronPython? The problem with that is compatibility with, and easy access to, existing libraries which is the main reason to use Python in the first place. I also think some of the criticisms in the GP comment are not accurate. most of the valuable libraries are native compiled? Some important ones are, but not all. I think a lot of the problem is that Python's usage has changed. Its great for a wide range of uses (scripting, web apps and other server stuff, even GUIs) but its really not a great match for scientific computing and the like but has become widely used there because it is easy to learn (and has lots of libraries for that now!).
braza 2y ago
> Too many fractured packaging solutions, not a single one well designed. And they all conflict. 100% this. Last 4 years, one of the most frustrating parts of SWE that I need to deal with on a daily basis is packaging data science & machine learning applications and APIs in Python. Maybe this is a very mid-solution, but one solution that I found was to use dockerized local environments with all dependencies pinned via Poetry [1]. The start setup is not easy, but now using some other Make file, it's something that I take only 4 hours with a DS to explain and run together and save tons of hours of in debugging and dependency conflict. > Python developers would rather save 1 line of code in the moment, even if it pushes the complexity off to another part of the system. Sounds odd to me in several projects that I worked on that folks bring the entire dependency on Scikit-Learn due to the train_test_split function [2] because the team thought that it would be simpler and easier to write a function that splits the dataset. [1] - https://github.com/orgs/python-poetry/discussions/1879 https://github.com/orgs/python-poetry/discussions/1879 [2] - https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.train_test_split.html https://scikit-learn.org/1.5/modules/generated/sklearn.model...
- biztos 2y ago
  I'm trying to do the same but with uv instead of poetry. So far so good, and it helps that for me delivering as a docker container is a requirement, but I have no idea what's going to happen if I need to run "real" ML stuff. (Just doing a lot of plotting so far.)

aaroninsf 2y ago

I agree with all of these and it makes me wonder as I do from time to time, has anyone managed to make a viable P#, a clean break which retains most of what most people love about the language and environment; and cheerfully asserts new and immutable change in things like <the technical parts of the above>. When I have looked into this it seems people can't help but improve one-more-thing or one-other-thing and end up just enjoying vaguely-pythonic language design.

aaroninsf 2y ago
Googling P# led me to this delight which is 100% unrelated: https://couragetotremble.blog/2007/08/09/p-language/ https://couragetotremble.blog/2007/08/09/p-language/
graemep 2y ago
IronPython? The problem with that is compatibility with, and easy access to, existing libraries which is the main reason to use Python in the first place. I also think some of the criticisms in the GP comment are not accurate. most of the valuable libraries are native compiled? Some important ones are, but not all. I think a lot of the problem is that Python's usage has changed. Its great for a wide range of uses (scripting, web apps and other server stuff, even GUIs) but its really not a great match for scientific computing and the like but has become widely used there because it is easy to learn (and has lots of libraries for that now!).

braza 2y ago

> Too many fractured packaging solutions, not a single one well designed. And they all conflict. 100% this. Last 4 years, one of the most frustrating parts of SWE that I need to deal with on a daily basis is packaging data science & machine learning applications and APIs in Python. Maybe this is a very mid-solution, but one solution that I found was to use dockerized local environments with all dependencies pinned via Poetry [1]. The start setup is not easy, but now using some other Make file, it's something that I take only 4 hours with a DS to explain and run together and save tons of hours of in debugging and dependency conflict. > Python developers would rather save 1 line of code in the moment, even if it pushes the complexity off to another part of the system. Sounds odd to me in several projects that I worked on that folks bring the entire dependency on Scikit-Learn due to the train_test_split function [2] because the team thought that it would be simpler and easier to write a function that splits the dataset. [1] - https://github.com/orgs/python-poetry/discussions/1879 https://github.com/orgs/python-poetry/discussions/1879 [2] - https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.train_test_split.html https://scikit-learn.org/1.5/modules/generated/sklearn.model...

biztos 2y ago
I'm trying to do the same but with uv instead of poetry. So far so good, and it helps that for me delivering as a docker container is a requirement, but I have no idea what's going to happen if I need to run "real" ML stuff. (Just doing a lot of plotting so far.)