4 ms·

I don't have any issues with Python's packaging ecosystem anymore, having settled comfortably into a pyenv+virtualenv+pip-tools as my "stack" after going around

I don't have any issues with Python's packaging ecosystem anymore, having settled comfortably into a pyenv+virtualenv+pip-tools as my "stack" after going around the block a few times. But even so, I must recognise how awful the experience is for new users. It's taken me years to settle into this system, and it can take half a day to get someone up to speed with these tools if they haven't used them. I also work a lot with non-developers who need to use or contribute to Python models, so that doesn't help — but I bet it would take an order of magnitude less time to get them up to speed with something like cargo. Coaching them has helped me see how user-hostile the process is to beginners. It also doesn't help how infectious "all-in-one" Python distributions like Anaconda can be, to the point that whenever anyone has an unexpected issue one of my first reflexes is to check their PATH. The fact that Rust has a widespread default toolchain multiplexer completely solves this issue. I appreciate that maintaining such a toolchain is work, and there is value as well in the diversity and choice of an open ecosystem. But perhaps building is one place where first-class support by the reference implementation creates a worthwhile tradeoff.

systems 3y ago
why do sometimes people say things like "and it can take half a day to get someone up to speed with these tools if they haven't used them" half a day is like almost no time at all half a day is just few hours, how is this a long time .. how is this any time at all makes me doubt myself a bit, am i too mediocre to think that way the previous line make more sense to me "It's taken me years to settle into this system" , now this is more like it
- carbotaniuman 3y ago
  I don't think it's half a day to get proficient, it's half a day to hack something half working together so they're unblocked and can do the other stuff they want to do.
  - throwawaymaths 3y ago
    And then another half a day to get pissed off because you don't remember what settings are/are not in your venv and trying to exit and/or get back into the venv (assuming no prior experience with venv) Fundamentally venv breaks your conceptions of what a shell is through cleverness, and that's a problem for people who are new.
  - deleted 3y ago
    [deleted]
- codeflo 3y ago
  Half a day to get something running that's a "one off" for you is insane. With a compiled language project, I'd download a binary. In Python, I need to reproduce the developer's setup. And I've yet to find two different Python projects where the official build instructions are compatible -- each one recommends a different environment virtualizer, a different runtime, different settings, and different C libraries that aren't part of the virtualized environment.
- earthboundkid 3y ago
  The thing that drives me insane about Python packaging is the degree of Stockholm Syndrome. PYTHON PACKAGING HAS KIDNAPPED YOU. YOU DON’T ACTUALLY LOVE IT. None of Go, JavaScript, or Rust require a half a day to figure out packaging. Don’t defend Python. Push for it to change or leave it behind.
- mattgreenrocks 3y ago
  Half a day is an eternity when it is makework. Trudging through old SO answers, bad docs, GitHub issues, overly enthusiastic blog posts by someone making a todo app, and, the coup de gras: hitting paywalls on the one tutorial that might help you. Factor in much higher rates for more senior devs and it amounts to a lot of waste.
- Aerbil313 3y ago
  This whole thread is a discussion of ease vs. simplicity. Easiness is subjective, simplicity is not. We should not compare tools in terms of their ease of use. There was a famous talk on this, maybe someone can link.
kevin_thibedeau 3y ago
I have issues precisely because of the misguided preference for virtualenvs in favor of traditional system package installation. It's obnoxious that pip now admonishes you for installing into site-packages even on a Debian system where that can't cause massive breakage. When you need isolated containers it's great. Everyone doesn't need a webdev focused, reproducible build for everyday shell life.
- dcow 3y ago
  You really shouldn’t, though. If you use a dependency manager for some deps you should use it for all deps. Using a global/system cache would be great if dependencies were versioned and each script could specify which version is needed, but they’re not to my knowledge. And it’s all fun and games until some random install script somewhere updates a global dep and your stuff breaks and you don’t know where to even begin looking.
  - Aerbil313 3y ago
    I’m still waiting for Nix to mature and for its UX to improve.
- movpasd 3y ago
  In my personal experience, it's absolutely necessary. Breaking changes are all over the place. I have non-dev coworkers who have built Python tools without any knowledge of package management, and it's a minefield getting it up and running. For individualised shell usage, sure. I have global installations of common data science utilities like pandas and jupyter, or requests. Reproducibility isn't just about deployment, it's also about coordination with colleagues.
dcow 3y ago
As someone new to using Python professionally after having used it here and there over the course of 15+ years, I’ve run into exactly this problem. It’s pretty standard for a language these days to bundle the dependency manager and build tooling. Python still does this via shell infection. And since there’s 5 different ways to do it it can leave someone trying to figure out what the right vibe is in 2023 spending hours reading about the pros and cons of everything. And all that just to land back on venv+pip+requirements.txt. Python needs a cargo. Is Poetry it? I’ve been meaning to try it…
- impulser_ 3y ago
  Have you tried Rye? https://github.com/mitsuhiko/rye https://github.com/mitsuhiko/rye This is probably the best package manager I used for Python. It feels a lot like Cargo. It sticks to the standards of Python. No custom lock files ect. Uses prebuilt Python so you don't have to build it. Handles global installs easily.
  - movpasd 3y ago
    This is very interesting indeed! A lot of the design choices fix issues I've also personally encountered. The "experimental" dissuades me from using it for real projects, but I'll be keeping an eye on it.
  - js2 3y ago
    > Correctly installed, rye will automatically pick up the right Python without manually activating the virtualenv. That is enabled by having ~/.rye/shims at higher priority in your PATH. I dislike this pattern. I don't want every language/tool manager in my PATH all the time. I much prefer the direnv. I hook only direnv into my shell. Then I can do what I need with an `.envrc` in each project directory.
    - dcow 3y ago
      I hate shell infection because 1) it only works in the shell, 2) it’s not stateless, and 3) depends on your working directory. I mean there’s a reason it’s called “shell infection”. Sounds like Rye specifically aims to address that issue. To each their own, I guess.

systems 3y ago

why do sometimes people say things like "and it can take half a day to get someone up to speed with these tools if they haven't used them" half a day is like almost no time at all half a day is just few hours, how is this a long time .. how is this any time at all makes me doubt myself a bit, am i too mediocre to think that way the previous line make more sense to me "It's taken me years to settle into this system" , now this is more like it

carbotaniuman 3y ago
I don't think it's half a day to get proficient, it's half a day to hack something half working together so they're unblocked and can do the other stuff they want to do.
- throwawaymaths 3y ago
  And then another half a day to get pissed off because you don't remember what settings are/are not in your venv and trying to exit and/or get back into the venv (assuming no prior experience with venv) Fundamentally venv breaks your conceptions of what a shell is through cleverness, and that's a problem for people who are new.
- deleted 3y ago
  [deleted]
codeflo 3y ago
Half a day to get something running that's a "one off" for you is insane. With a compiled language project, I'd download a binary. In Python, I need to reproduce the developer's setup. And I've yet to find two different Python projects where the official build instructions are compatible -- each one recommends a different environment virtualizer, a different runtime, different settings, and different C libraries that aren't part of the virtualized environment.
earthboundkid 3y ago
The thing that drives me insane about Python packaging is the degree of Stockholm Syndrome. PYTHON PACKAGING HAS KIDNAPPED YOU. YOU DON’T ACTUALLY LOVE IT. None of Go, JavaScript, or Rust require a half a day to figure out packaging. Don’t defend Python. Push for it to change or leave it behind.
mattgreenrocks 3y ago
Half a day is an eternity when it is makework. Trudging through old SO answers, bad docs, GitHub issues, overly enthusiastic blog posts by someone making a todo app, and, the coup de gras: hitting paywalls on the one tutorial that might help you. Factor in much higher rates for more senior devs and it amounts to a lot of waste.
Aerbil313 3y ago
This whole thread is a discussion of ease vs. simplicity. Easiness is subjective, simplicity is not. We should not compare tools in terms of their ease of use. There was a famous talk on this, maybe someone can link.

kevin_thibedeau 3y ago

I have issues precisely because of the misguided preference for virtualenvs in favor of traditional system package installation. It's obnoxious that pip now admonishes you for installing into site-packages even on a Debian system where that can't cause massive breakage. When you need isolated containers it's great. Everyone doesn't need a webdev focused, reproducible build for everyday shell life.

dcow 3y ago
You really shouldn’t, though. If you use a dependency manager for some deps you should use it for all deps. Using a global/system cache would be great if dependencies were versioned and each script could specify which version is needed, but they’re not to my knowledge. And it’s all fun and games until some random install script somewhere updates a global dep and your stuff breaks and you don’t know where to even begin looking.
- Aerbil313 3y ago
  I’m still waiting for Nix to mature and for its UX to improve.
movpasd 3y ago
In my personal experience, it's absolutely necessary. Breaking changes are all over the place. I have non-dev coworkers who have built Python tools without any knowledge of package management, and it's a minefield getting it up and running. For individualised shell usage, sure. I have global installations of common data science utilities like pandas and jupyter, or requests. Reproducibility isn't just about deployment, it's also about coordination with colleagues.

dcow 3y ago

As someone new to using Python professionally after having used it here and there over the course of 15+ years, I’ve run into exactly this problem. It’s pretty standard for a language these days to bundle the dependency manager and build tooling. Python still does this via shell infection. And since there’s 5 different ways to do it it can leave someone trying to figure out what the right vibe is in 2023 spending hours reading about the pros and cons of everything. And all that just to land back on venv+pip+requirements.txt. Python needs a cargo. Is Poetry it? I’ve been meaning to try it…

impulser_ 3y ago
Have you tried Rye? https://github.com/mitsuhiko/rye https://github.com/mitsuhiko/rye This is probably the best package manager I used for Python. It feels a lot like Cargo. It sticks to the standards of Python. No custom lock files ect. Uses prebuilt Python so you don't have to build it. Handles global installs easily.
- movpasd 3y ago
  This is very interesting indeed! A lot of the design choices fix issues I've also personally encountered. The "experimental" dissuades me from using it for real projects, but I'll be keeping an eye on it.
- js2 3y ago
  > Correctly installed, rye will automatically pick up the right Python without manually activating the virtualenv. That is enabled by having ~/.rye/shims at higher priority in your PATH. I dislike this pattern. I don't want every language/tool manager in my PATH all the time. I much prefer the direnv. I hook only direnv into my shell. Then I can do what I need with an `.envrc` in each project directory.
  - dcow 3y ago
    I hate shell infection because 1) it only works in the shell, 2) it’s not stateless, and 3) depends on your working directory. I mean there’s a reason it’s called “shell infection”. Sounds like Rye specifically aims to address that issue. To each their own, I guess.