4 ms·

If you don’t care which specific version of python you are using, do not use pyenv. You will know when you care. These days, doing common tasks, the constraint

by b33j0r 3y ago

If you don’t care which specific version of python you are using, do not use pyenv. You will know when you care. These days, doing common tasks, the constraint solver in poetry will often tell you: “Hey! I can’t find a version of sentencepiece with metadata that lets me combine this with python 3.12.2. Sorry! I give up!” Now, if you aren’t concerned with using your bare metal for CUDA or fancy new MPS or AMD stuff. Just ignore this and use containers. I’d use podman compose. However, I use pyenv on every machine. Because it compiles specific versions I actually need, even to create various virtual environments. If compiling python automatically sounds tough, you probably don’t need to anyway. To describe the problem you’d see. I try to use poetry by default, though I think it became popular before it was PEP-compliant or useful in devops. It is impossible to control the behavior of other package managers, and poetry is/was strict about that. Which means you can’t force deploy in many cases. (Better lately.) For the problem pyenv helps to solve, I back-up my pyproject.toml setuptools backend with pip and requirements.txt. These days, requirements-cuda.txt and requirements-mps.txt. The landscape is still a disaster for binary compatibility, but it can be done lol. (I’ve been doing python packaging professionally since prom, which was python 2.6 give or take.)

elevation 3y ago
This kind of error surprises me: > I can’t find a version of sentencepiece with metadata that lets me combine this with python 3.12.2 I'm a reasonably advanced python user. I've shipped web apps, Desktop GUIs, cli-tools, and even written cpython extension modules for custom hardware control. I typically target the system `python3` of whatever linux distribution I'm shipping to and I use the system 'python3-virtualenv' for venvs. But I have never encountered a dependency resolution issue or been forced to use poetry. What am I doing wrong?
- b33j0r 3y ago
  (Edit2 tl:dr: it’s that other package maintainers don’t always keep up with new semver constants in their metadata, particularly on anything cutting-edge) It’s always the inclusion of a specific dependency we added for a feature, and based on that dev’s knowledge and experience. It’s often me, but not always. This doesn’t happen in ecosystems with a base package versioning. This is arguably why anaconda became popular, and why we target base docker images of ubuntu. Doesn’t work in complex deployments based on money rather than ideals, every time. At least in my career. Edit: first time I dealt with this, we ended up forking a dependency chain instead of using pip. I lost that war and they ended up reviving a legacy PHP app instead of funding python dev.
- Hackbraten 3y ago
  Sometimes, an app or library developer wants to target multiple Linux distributions. For example, I want my own Python apps to work equally well on Debian oldstable (which currently provides Python 3.9) and Arch Linux (currently on 3.11). That means that I’m going to choose the lowest common version (3.9) as the language level for my app or library. And if I program against a 3.9 language level, I absolutely *refuse* to use any other interpreter than a 3.9 one at development time. (If I used a newer one, then my linter and type checker would helpfully give me false alarms all the time, unaware of the actual language level.) Hence, I use pyenv and poetry to get exactly the interpreter for the language level I want, and to allow pylint and mypy to be perfectly aware of the language level I’m using at development time, allowing them to give me findings that actually make sense.
- dragonwriter 3y ago
  > But I have never encountered a dependency resolution issue or been forced to use poetry. What am I doing wrong? Not using python on a system without the necessary build tools while trying to use dependencies with native code that don't have binaries built for your combination of python version and platform? This used to be a huge problem with Python on Windows, as things that weren't pure Python would very often not have binary packages or have them for a narrow set of python versions, often not the same set as other dependencies. (Not just a windows problem, but it was definitely big on windows.) Tooling and practices have advanced so that more packages are automatically built against a wider set of targets, so the problem is a lot smaller than it used to be.

by b33j0r 3y ago

elevation 3y ago

This kind of error surprises me: > I can’t find a version of sentencepiece with metadata that lets me combine this with python 3.12.2 I'm a reasonably advanced python user. I've shipped web apps, Desktop GUIs, cli-tools, and even written cpython extension modules for custom hardware control. I typically target the system `python3` of whatever linux distribution I'm shipping to and I use the system 'python3-virtualenv' for venvs. But I have never encountered a dependency resolution issue or been forced to use poetry. What am I doing wrong?

b33j0r 3y ago
(Edit2 tl:dr: it’s that other package maintainers don’t always keep up with new semver constants in their metadata, particularly on anything cutting-edge) It’s always the inclusion of a specific dependency we added for a feature, and based on that dev’s knowledge and experience. It’s often me, but not always. This doesn’t happen in ecosystems with a base package versioning. This is arguably why anaconda became popular, and why we target base docker images of ubuntu. Doesn’t work in complex deployments based on money rather than ideals, every time. At least in my career. Edit: first time I dealt with this, we ended up forking a dependency chain instead of using pip. I lost that war and they ended up reviving a legacy PHP app instead of funding python dev.
Hackbraten 3y ago
Sometimes, an app or library developer wants to target multiple Linux distributions. For example, I want my own Python apps to work equally well on Debian oldstable (which currently provides Python 3.9) and Arch Linux (currently on 3.11). That means that I’m going to choose the lowest common version (3.9) as the language level for my app or library. And if I program against a 3.9 language level, I absolutely *refuse* to use any other interpreter than a 3.9 one at development time. (If I used a newer one, then my linter and type checker would helpfully give me false alarms all the time, unaware of the actual language level.) Hence, I use pyenv and poetry to get exactly the interpreter for the language level I want, and to allow pylint and mypy to be perfectly aware of the language level I’m using at development time, allowing them to give me findings that actually make sense.
dragonwriter 3y ago
> But I have never encountered a dependency resolution issue or been forced to use poetry. What am I doing wrong? Not using python on a system without the necessary build tools while trying to use dependencies with native code that don't have binaries built for your combination of python version and platform? This used to be a huge problem with Python on Windows, as things that weren't pure Python would very often not have binary packages or have them for a narrow set of python versions, often not the same set as other dependencies. (Not just a windows problem, but it was definitely big on windows.) Tooling and practices have advanced so that more packages are automatically built against a wider set of targets, so the problem is a lot smaller than it used to be.