5 ms·

The thing that put me off of Rye is that it strongly encourages you to use non-standard Python builds. From their philosophy page: https://rye.astral.sh/philos

The thing that put me off of Rye is that it strongly encourages you to use non-standard Python builds. From their philosophy page: https://rye.astral.sh/philosophy/ https://rye.astral.sh/philosophy/ > No Python Binary Distributions: CPython builds from python.org are completely inadequate. On some platforms you only get an .msi installer, on some you literally only get tarballs. The various Python distributions that became popular over the years are diverging greatly and cause all kinds of nonsense downstream. This is why this Project uses the indygreg standalone builds. I hope that with time someone will start distributing well maintained and reliable Python builds to replace the mess we are dealing with today. And here is info about those particular indygreg builds. https://gregoryszorc.com/docs/python-build-standalone/main/ https://gregoryszorc.com/docs/python-build-standalone/main/ It is, however, possible to choose a different Python. https://rye.astral.sh/guide/toolchains/ https://rye.astral.sh/guide/toolchains/ I've never really experienced the problem they are describing. Any official Python build I've gotten from python.org has worked. Every normal old Python I've gotten from my distribution's package manager has worked. Every Python included with an official Docker container image has worked. I'm sure their special builds will also work just fine, but their non-standardness gives me pause. There's even a list of behavior quirks. Why are we adding quirks to something that has no problems? And the fact that the rye philosophy seems so intent on using them, turns me off from it compared to the alternatives that sensibly default to using the Python you already have available on your OS.

p5a0u9l 2y ago
Agree. Support PSF, don’t advocate a new “right way” of doing it. Weird position.
- akdlfa 2y ago
  The PSF is the marketing and self-promotion arm in the Python ecosystem. It allows people who have done little or nothing to gain influence. The PSF has nothing to do with software development.
- cozzyd 2y ago
  UV is more likely to support SPF
  - OJFord 2y ago
    Those French splitters!
oconnor663 2y ago
I'm just guessing, but I imagine the scenario goes like this: 1. Work at a company that runs some ancient version of CentOS or something. 2. The system package manager doesn't have anything newer than Python 3.6. 3. Official binaries from python.org print an inscrutable error about the glibc version when you try to run them. 4. Building Python from source requires some new system dependencies, takes forever, and then produces an even less scrutable error about a missing header file. 5. Googling that error leads to a ./configure flag that works around the issue, which upstream GCC fixed in 2017. 6. Success? If you haven't seen that error #3 before, or dealt with "manylinux" build environments, you've dodged an entire world of pain. The same goes for scripting installs on Windows, and for the part of that page that's talking about "limiting the CPU instructions that can be used" :')
- noisy_boy 2y ago
  > If you haven't seen that error #3 before I have been in the #3 hell, almost exactly how you described but it was always about SSL and its missing headers. On my desktop wiki, the most important section about Python is the one that contains the incantations required to compile SSL, setting myriad of variables and using pyenv to build a newer (3.10/3.11/3.12) Python.
  - swells34 2y ago
    I've run into #3 quite often in embedded Linux projects, especially when dealing with the Jetson ecosystem where upgrading to a modern Python is a nightmare due to all the specialized hardware. Glad to see I'm not the only one who runs into this.
- staticautomatic 2y ago
  The scenario might also go something like this: 1. Try to install pytorch
  - mft_ 2y ago
    Or spend days (literally) trying to install OpenCV...
metadat 2y ago
I regularly download the python source code, compile it with standard prod optimizations, then install to /use/local/python${version}. This works extremely consistently since python 3.7 (released in 2018). In my experience, these commands are so stable and consistent they could be automated away. What might the author's issue or underlying protest be?
- Hamuko 2y ago
  Doesn't pyenv basically do that automating away? I don't think I've ever had issues compiling Python using pyenv.
  - metadat 2y ago
    No idea what that is, the standard distribution is so easy to work with I don't need anything else.
  - rat87 2y ago
    I've had a ton of issues with pyenv and using pyenv/pyenv-virtualenv/poetry that were fixed by moving to rye
  - the_mitsuhiko 2y ago
    There is also a world of windows out there. Compiling Python from scratch is not a ton of fun there. Rye wants to have a very consistent experience for everybody.
    - robertlagrant 2y ago
      One of the key benefits of Python is it's very easy to motivate for getting a Mac or Linux laptop from IT when you use it :)
      - krageon 2y ago
        Python is a nightmare to use on those platforms as well. It's one of my favourite languages, but I have (until now I guess) started avoiding it like the plague for anything important or distributable because of the actual hell that is dealing with it in production.
yellowapple 2y ago
I do like that Rye's Python builds apparently use libedit instead of readline. One less GPL-licensed thing to worry about for redistribution :)
- rat87 2y ago
  As someone with a large project that depends on the standard readline that was a major hiccup when moving to rye. Luckily there's a gnureadline package.
selcuka 2y ago
> No Python Binary Distributions: CPython builds from python.org are completely inadequate. On some platforms you only get an .msi installer, on some you literally only get tarballs. I'm just guessing, but they could mean that there are no macOS/Windows binaries for security releases of older Python versions. You can't, for example, download an installer for Python 3.10.14. The last Windows installer is for Python 3.10.11 (April 5, 2023).
flurie 2y ago
Python building, packaging and deployment has two extreme states: the king's highway and the hall of a thousand knives. If the portable Python suggestions do not make sense to you, then consider yourself lucky, because you have managed to stick to the highway.
- ttyprintk 2y ago
  Best of the thousand knives I think is `shiv`. Produces one big artifact which self-installs without colliding with other runtimes on the system.
the_mitsuhiko 2y ago
Original author of Rye here: there are no official Python builds outside of macOS and Windows and the macOS builds cannot be installed programmatically. They also cannot be placed in any location other than the dedicated framework location which often causes issues for people who do not expect specific versions to be installed. Quite often installing the macOS builds of Python breaks other tools that expect a different version to be there. I’m glad regular Python versions work for you, and you can register them with Rye. That’s very intentionally supported. The goal of rye is to reach higher. It wants to see how close we can get to an experience you can get from node or Rust. That includes getting access to Python itself. I have been programming Python since 2.2 and have helped countless of people over the years with their Python challenges. From mentoring to tutoring to just helping on IRC and other places. So many times people had a completely broken Python installation or ran into compilation errors. Even pyenv ships patches to make Python compile cleanly in some cases. The indygreg builds have issues, no doubt about it. In an ideal world the PSF would distribute portable Python builds for all platforms.
crabbone 2y ago
The way different Pythons from PyPI don't work can be, for example that various optional modules are (not) included. For example, for my own Python installs, I build without tkinter. I have no use for this module, and it's always a burden to find the outdated XWidget libraries necessary for this pile of garbage to build. Seeing how this module is basically useless, a lot of Linux distros also exclude it from the Python they package. But PyPI builds try to include everything. There are few more modules like that. Another aspect is various defaults that help Python locate its parts or control its loading. Eg. the site module or sysconf. For various reasons various vendors may configure these differently. This may result in some packages being broken upon install. I.e. Python programmers are notoriously bad at creating packages and understanding how they work (also, Wheel format is a dumpster fire of nonsense). So, a lot of developers don't understand the consequences of packaging anything that's not strictly Python source code (which, frankly, should never have been packaged! but hey, remember, Wheel? dumpster fire? So... where was I...) anyways, native libraries packaged with python source may end up in some random place Python doesn't look at, and consequently fails to load, or other data files end up in the wrong place because the developer packaging them after countless trial and error has made it work on their computer, with their particular Python installation, but they still don't know why it worked. Similarly, if a package wants to provide header files so that other packages can compile against the native modules the package provides... oh-ho-ho, bet you didn't know that's possible, right?! Well, don't bother. It mostly doesn't work anyways. And there's a bunch more stuff like that. As a "typical" user, you might have never encountered any of the issues above, but me, as a Python infra person who's summoned to extinguish fires started by talented Python programmers by using tools like the one in OP deals with this stuff pretty regularly. So, I can relate to the disappointment with any aspect of Python infrastructure. There has never been a case of me discovering something in Python infra and looking at it with admiration. At best it's passable, but in most instances it's hilariously bad.
WhyNotHugo 2y ago
I tried Rye during its first days. It would (without any indication) download some custom build of Python, which was dynamically linked so won't work unless you're running a distribution similar to the build environment. Linux distributions ARE NOT binary compatible, you can't just build Python on one distro and run in on another. You need to statically link everything to do that.

p5a0u9l 2y ago

Agree. Support PSF, don’t advocate a new “right way” of doing it. Weird position.

akdlfa 2y ago
The PSF is the marketing and self-promotion arm in the Python ecosystem. It allows people who have done little or nothing to gain influence. The PSF has nothing to do with software development.
cozzyd 2y ago
UV is more likely to support SPF
- OJFord 2y ago
  Those French splitters!

oconnor663 2y ago

I'm just guessing, but I imagine the scenario goes like this: 1. Work at a company that runs some ancient version of CentOS or something. 2. The system package manager doesn't have anything newer than Python 3.6. 3. Official binaries from python.org print an inscrutable error about the glibc version when you try to run them. 4. Building Python from source requires some new system dependencies, takes forever, and then produces an even less scrutable error about a missing header file. 5. Googling that error leads to a ./configure flag that works around the issue, which upstream GCC fixed in 2017. 6. Success? If you haven't seen that error #3 before, or dealt with "manylinux" build environments, you've dodged an entire world of pain. The same goes for scripting installs on Windows, and for the part of that page that's talking about "limiting the CPU instructions that can be used" :')

noisy_boy 2y ago
> If you haven't seen that error #3 before I have been in the #3 hell, almost exactly how you described but it was always about SSL and its missing headers. On my desktop wiki, the most important section about Python is the one that contains the incantations required to compile SSL, setting myriad of variables and using pyenv to build a newer (3.10/3.11/3.12) Python.
- swells34 2y ago
  I've run into #3 quite often in embedded Linux projects, especially when dealing with the Jetson ecosystem where upgrading to a modern Python is a nightmare due to all the specialized hardware. Glad to see I'm not the only one who runs into this.
staticautomatic 2y ago
The scenario might also go something like this: 1. Try to install pytorch
- mft_ 2y ago
  Or spend days (literally) trying to install OpenCV...

metadat 2y ago

I regularly download the python source code, compile it with standard prod optimizations, then install to /use/local/python${version}. This works extremely consistently since python 3.7 (released in 2018). In my experience, these commands are so stable and consistent they could be automated away. What might the author's issue or underlying protest be?

Hamuko 2y ago
Doesn't pyenv basically do that automating away? I don't think I've ever had issues compiling Python using pyenv.
- metadat 2y ago
  No idea what that is, the standard distribution is so easy to work with I don't need anything else.
- rat87 2y ago
  I've had a ton of issues with pyenv and using pyenv/pyenv-virtualenv/poetry that were fixed by moving to rye
- the_mitsuhiko 2y ago
  There is also a world of windows out there. Compiling Python from scratch is not a ton of fun there. Rye wants to have a very consistent experience for everybody.
  - robertlagrant 2y ago
    One of the key benefits of Python is it's very easy to motivate for getting a Mac or Linux laptop from IT when you use it :)
    - krageon 2y ago
      Python is a nightmare to use on those platforms as well. It's one of my favourite languages, but I have (until now I guess) started avoiding it like the plague for anything important or distributable because of the actual hell that is dealing with it in production.

yellowapple 2y ago

I do like that Rye's Python builds apparently use libedit instead of readline. One less GPL-licensed thing to worry about for redistribution :)

rat87 2y ago
As someone with a large project that depends on the standard readline that was a major hiccup when moving to rye. Luckily there's a gnureadline package.

selcuka 2y ago

> No Python Binary Distributions: CPython builds from python.org are completely inadequate. On some platforms you only get an .msi installer, on some you literally only get tarballs. I'm just guessing, but they could mean that there are no macOS/Windows binaries for security releases of older Python versions. You can't, for example, download an installer for Python 3.10.14. The last Windows installer is for Python 3.10.11 (April 5, 2023).

flurie 2y ago

Python building, packaging and deployment has two extreme states: the king's highway and the hall of a thousand knives. If the portable Python suggestions do not make sense to you, then consider yourself lucky, because you have managed to stick to the highway.

ttyprintk 2y ago
Best of the thousand knives I think is `shiv`. Produces one big artifact which self-installs without colliding with other runtimes on the system.

the_mitsuhiko 2y ago

Original author of Rye here: there are no official Python builds outside of macOS and Windows and the macOS builds cannot be installed programmatically. They also cannot be placed in any location other than the dedicated framework location which often causes issues for people who do not expect specific versions to be installed. Quite often installing the macOS builds of Python breaks other tools that expect a different version to be there. I’m glad regular Python versions work for you, and you can register them with Rye. That’s very intentionally supported. The goal of rye is to reach higher. It wants to see how close we can get to an experience you can get from node or Rust. That includes getting access to Python itself. I have been programming Python since 2.2 and have helped countless of people over the years with their Python challenges. From mentoring to tutoring to just helping on IRC and other places. So many times people had a completely broken Python installation or ran into compilation errors. Even pyenv ships patches to make Python compile cleanly in some cases. The indygreg builds have issues, no doubt about it. In an ideal world the PSF would distribute portable Python builds for all platforms.

crabbone 2y ago

The way different Pythons from PyPI don't work can be, for example that various optional modules are (not) included. For example, for my own Python installs, I build without tkinter. I have no use for this module, and it's always a burden to find the outdated XWidget libraries necessary for this pile of garbage to build. Seeing how this module is basically useless, a lot of Linux distros also exclude it from the Python they package. But PyPI builds try to include everything. There are few more modules like that. Another aspect is various defaults that help Python locate its parts or control its loading. Eg. the site module or sysconf. For various reasons various vendors may configure these differently. This may result in some packages being broken upon install. I.e. Python programmers are notoriously bad at creating packages and understanding how they work (also, Wheel format is a dumpster fire of nonsense). So, a lot of developers don't understand the consequences of packaging anything that's not strictly Python source code (which, frankly, should never have been packaged! but hey, remember, Wheel? dumpster fire? So... where was I...) anyways, native libraries packaged with python source may end up in some random place Python doesn't look at, and consequently fails to load, or other data files end up in the wrong place because the developer packaging them after countless trial and error has made it work on their computer, with their particular Python installation, but they still don't know why it worked. Similarly, if a package wants to provide header files so that other packages can compile against the native modules the package provides... oh-ho-ho, bet you didn't know that's possible, right?! Well, don't bother. It mostly doesn't work anyways. And there's a bunch more stuff like that. As a "typical" user, you might have never encountered any of the issues above, but me, as a Python infra person who's summoned to extinguish fires started by talented Python programmers by using tools like the one in OP deals with this stuff pretty regularly. So, I can relate to the disappointment with any aspect of Python infrastructure. There has never been a case of me discovering something in Python infra and looking at it with admiration. At best it's passable, but in most instances it's hilariously bad.

WhyNotHugo 2y ago

I tried Rye during its first days. It would (without any indication) download some custom build of Python, which was dynamically linked so won't work unless you're running a distribution similar to the build environment. Linux distributions ARE NOT binary compatible, you can't just build Python on one distro and run in on another. You need to statically link everything to do that.