5 ms·
Python as a language is nice. Python's version and package management is nothing short of a nightmare.
by smcleod 1y ago
Python as a language is nice. Python's version and package management is nothing short of a nightmare.
- est 1y agoPython's package management is OK. The main culprit is .so libraries Think JNI or cgo management.
- t43562 1y agoYes, I've never really understood the complaint about python packaging - building native code is not something that is ever easy to guarantee across multiple distributions and operating systems. Those native packages can be in any language and require any odd combination of tools to build. Who has truly solved that problem?
- nikisweeting 1y agowheels, conda, docker, firecracker, unikernel, flatpak
- t43562 1y agoflatpak, docker ? i.e. include an almost complete distribution just to make one package work? but what if you want that package to work in the distribution you have now? What if you need 2 different packages that are in different flatpacks or different docker images?
- staunton 1y agoSet them up, each in its own container, and use GRPC to communicate between them... (Not even kidding, I've seen people do this)
- nikisweeting 1y agoobviously don't try those first I'm just saying there's a stack of options to work with for any level of isolation you desire. start with wheels if you just want pre-built binaries, move up to conda if you need arch and OS-specific C libs that depend on system packages, flatpack or docker if you dont want those to mess up your host, or unikernel/firecracker/VMs if you need kernel modules or hardware virtualization.
- Orygin 1y agoIf you don't need to link C lib, you can build any combination of arch and OS for a golang program. The default tooling allows you to do so easily. If you need to link a C lib, there are ways to set it up to compile other OS (and maybe other archs).
- est 1y agotake numpy as an example, it's gfortran mixed with C. How does cgo handle that? And there's ffmpeg....
- Orygin 1y agoIf you got object files you can compile whatever if you have the compiler toolset. I have made a go program that links ffmpeg and is built from linux to Windows and Macos. It was not super easy but it's doable.
- RamblingCTO 1y agoI'll throw in BLAS and LAPACK. Fuck, what a nightmare it always has been to get scipy running. I've always ended up with a wild mix of conda and pip installed shit that just wouldn't work.
- blurbleblurble 1y agoIt is not okay (maybe uv fixes this?)
- est 1y agopure .py libs can be "vendorized" directly into project.
- jcattle 1y agoGoing to be honest: With uv it really isn't that bad anymore. Sure, the whole packaging ecosystem is still really bad, but uv abstracts over most of it. Unless you're doing some really esoteric stuff you'll be just fine.
- xyzsparetimexyz 1y agoUsing a nix devshell has worked out well for scripts for me so far. I haven't figured out what that workflow is like for larger projects though. I'm not interested in learning Uv
- smallerfish 1y agoThere's nothing to learn. Get Claude code to install uv into your devshell and cut over your requirements. 5 minutes.
- expenses3 1y agoAI? :vomit_emoji:
- __MatrixMan__ 1y agoOr do it yourself, 10 minutes
- smallerfish 1y agoNot being comfortable with AI isn't a valid reason for being intimidated by new tech. At this point any developer should be able to write a command line simple tool using a language that's brand new to them in a couple of hours. Simply using a new tool for the first time is trivial. Don't get left behind.
- expenses3 1y agoI'd rather work in a different field than be forced to use a LLM or vibe code all day.
- smallerfish 1y agoDo you feel that way about IDEs also? Or compilers?
- __MatrixMan__ 1y ago
- lazzlazzlazz 1y agoThis hasn't really been true for a while now. `uv` has radically improved the experience.
- cedws 1y agoThere’s still 20 years of projects using everything that became before uv. They didn’t upgrade the moment uv came into existence. Data science-land still uses other rubbish too.
- zimpenfish 1y ago> They didn’t upgrade the moment uv came into existence. There's also projects that can't use `uv` because it doesn't like their current `requirements.txt`[0] and I have no bandwidth to try and figure out how to work around it. [0] We have an install from `git+https` in there and it objects strongly for some reason. Internet searches have not revealed anything helpful.
- bootsmann 1y agoUnrelated to uv but the problem with having a git ref in requirements.txt is that pip will treat it as a fixed version so it will strictly narrow the other dependencies it can resolve, which gets exceptionally difficult to reason about once that packages also loads a package from another git ref. Throwing everything into codeartifact (or equivalent on other clouds) is better longterm.
- zanie 1y agoIf you open even a brief issue and tag me @zanieb I'm happy to take a look!
- zimpenfish 1y agoTa, will give that a go when I've got some free time.
- bborud 1y ago
- timw4mail 1y agoIt's like the (usually) interpreted equivalent to C/C++. There are lots of 'standard' package management choices. And it seems like the package resolution is finally local by default, although that requires a 'virtualenv', which seems to be a legacy of the global packaging system.
- __MatrixMan__ 1y agoI expect that for every two python users who know what `pip` is, there are three who are discouraged from thinking about the environment at all. Given that, focusing on the language and not the packaging is the right choice. Packaging is most often somebody else's problem. It's just HN users are more likely to be that somebody else. Probably we have to deal with non-python dependencies anyway so we're reaching for bigger hammers like docker or nix. It would be nice if there wasn't a mess of python package managers, but whichever one I use ends up being a somewhat unimportant middle-layer anyway.