7 ms·

I don't normally use Python. When I see something that I'd like to install, and the docs say "written in Python", my heart sinks. I know I'll have to find out h

by jwr 3y ago

I don't normally use Python. When I see something that I'd like to install, and the docs say "written in Python", my heart sinks. I know I'll have to find out how to install multiple dependencies in various packaging systems, all of which will be incompatible (but surely much better) than my system package manager. I find all of this ironic given that the main gripe people used to have against Common Lisp was "but how do I make a self-contained binary?". Somehow this was never required of Python.

bombolo 3y ago
For me if I can do apt install is good. No care which language it's written in.
- rgoulter 3y ago
  Sure, it's convenient if there's already a system package; in which case, yes, the language is irrelevant. However, if you want to run software which isn't available as a system package, then you'll need to know how to build/run the software.
  - roflyear 3y ago
    Isn't that true of many languages?
    - arp242 3y ago
      Kind of, but some languages are definitely easier to build than others (on average).
    - rgoulter 3y ago
      Surely "to run the program from source, you need to know how to build/run it" applies to every language. But I'm more comfortable working through problems I might encounter in languages I've worked with, than in languages I'm not familiar with. I'd say Python is notorious, since you're probably going to hear about at least six or seven ways people are able to run things; so if you're not familiar with it, you're likely to do something wrong. -- e.g. elsewhere in this thread, you've pointed out that stuff should be run from a venv; that's something you have to know.
      - roflyear 3y ago
        It's true: some people like Docker for Python, some like venv, some like conda... some lose it entirely and install everything locally! (acting like I haven't done this at least once!)
    - jeltz 3y ago
      Yes, but Python is among the harder languages to build while other languages can be a breeze. And that is despite having used Python much more than some of them.
      - bombolo 3y ago
        Python is rather easy. python3 -m venv bla; . bla/bin/activate; pip install -r requirements.txt; ./command Of course badly written code is buggy and will require bug fixing because it only works on the machine of the developer. I'm not sure this isn't true in other languages. I've seen go CLI utilities that were dynamically linking Gnome libraries and could not start without installing Gnome.
    - dagw 3y ago
      Many languages have much better tools that make it easy to compile/bundle an artefact that can be run without the end user having to build or install anything.
      - bombolo 3y ago
        Much better tools? Welcome to use C++ without a linux distribution providing compiler and headers!
- BiteCode_dev 3y ago
  Yes, if you are a user. But if you are a developer, it's not great.
theshrike79 3y ago
This was the main reason I started writing my random tools and scripts in Go. I can just write, cross-compile and deploy. Trying to get a "simple" Python tool with library dependencies working on a random server is a huge pain in the keister I'd rather not deal with if I have the option. This became even easier now that I can "simply" feed my old Python stuff to GPT4 and have it transform it to mostly-working Go code =)
- eurekin 3y ago
  Interesting. Where are those tools deployed to? Is there any runtime needed on the client? How can I grab a go tool on a pristine machine?
  - jbarrs 3y ago
    Not the original commenter, but Go programs are all statically-compiled, with everything needed at run time bundled into the executable. Go does use a runtime, but that's incorporated into the exe, so no external libraries required. As such, you can get a program written in Go running on a new machine just by downloading it :) For machines where you may actually have a Go installation already, the majority of programs now can also be installed from source via Go's built-in package manager, which installs it to your home directory. A lot of dev utils, like go-imports, the language server, etc. are typically installed this way. It's usually just a single command (though the exact command can vary between projects, but is usually signposted fairly clearly).
  - theshrike79 3y ago
    For 99% of stuff I need to move either one statically compiled Go binary and maybe a configuration yaml. That's it. With Python I need to have Python installed in there, and it has to be the correct version because of <reasons>. Then I need to figure out how I can install the dependencies for the program, with the correct version(s). This usually would require me to figure out what's the current virtual env du jour and how do I get it working on a random *nix distro on the target computer. Then I can install what's in requirements.txt or whatever the venv tool is using. And now I can maybe run it if all went well. And when I need to update said program, with Go I can just scp/rsync a new binary on top of the old one at it'll Just Work. With Python I need to move the Python source file(s) there, check for possible updates to libraries, update them using the venv tool and re-check that I have the correct Python version installed once more. Now picture a situation where I need to do the process above for, let's say 12 different servers on different clients after every fortnight. All servers are Intel-based, but run a variety of Linux distros. I can use the exact same Go binary for every single one, the Python solution is a complete clusterf... I do need to spend a bit more time when developing a Go solution compared to Python, but the ease of deployment is definitely worth it.
throwawaaarrgh 3y ago
Most programmers don't give a crap about the user. Python assumes that the user will be a Python programmer, and want to maintain a whole independent part of their system just to run a Python program.
jrib 3y ago
What do you normally use? I'm curious what context you are comparing with.
- jeltz 3y ago
  Perl, Ruby, JavaScript, PHP, Bash, Go, C, C++, Rust and even Java is easier to deal with than Python tools. Seeing that it is a Python tool usually make my heart drop a lot and makes me look for alternatives before attempting to get it running.
- wongarsu 3y ago
  I assume the gold standard is just downloading a compiled binary and using that. Which is the norm for almost everyone ... except python projects. When it comes to compiling/running from source, Python is in the middle of the pack. It's not as smooth as rust (install cargo from your package manager or from rustup, run "cargo build") but not as bad as C++ (run through the install section of the readme of each dependency, then debug whatever build system the project is using).
supriyo-biswas 3y ago
The lack of a way to produce a self contained binary is why I switched away from Python for a personal project of mine. However, to address the other part of your comment, pipx works quite well for installing command line applications.
- globular-toast 3y ago
  Why does it matter if a program is a self-contained binary? It seems like such an odd requirement. If you really want that it seems in principle easy to compile it all into one script with no imports and put a shebang at the top. What's the point, though? Pyinstaller is pretty good for if you need a standalone installer.
  - spicyusername 3y ago
    Self contained binaries are very easy to distribute and use.
  - bheadmaster 3y ago
    > Why does it matter if a program is a self-contained binary? Because simple is better than complex. I'd rather download a binary than download a binary + dependencies + set them up + bookkeep for when I want to delete all those files. > Pyinstaller is pretty good for if you need a standalone installer. PyInstaller doesn't work for many edge cases. If you're using a Python package that uses a compiled binary written in another language, good luck on your way down the rabbit hole of PyInstaller config. I personally could not succeed in packaging a uvicorn app, for example.
    - redsaber 3y ago
      > I personally could not succeed in packaging a uvicorn app, for example. I kinda did managed to run a fastapi with uvicorn and pyinstaller. And I did nothing special
      - bheadmaster 3y ago
roflyear 3y ago
pip installing one thing is usually not a problem. You can do the binary thing with Python for sure. Does Lisp really work much better? Part of the issue with Python is versions of the language. So you really shouldn't be installing packages locally in most cases, but you should use one of these tools so you're running your dev environment isolated from other environments.
- crabbone 3y ago
  > Does Lisp really work much better? Most of the popular ones are a lot better than Python. Python might be comparable to CAD Lisp or that abomination embedded in GIMP (though I think they stopped supporting it) in terms of quality of design and execution. > So you really shouldn't be installing packages locally in most cases Where do you suggest I install them instead? On someone else's computer? What kind of advise is that?..
- BiteCode_dev 3y ago
  If you read the article, it argues that yes, pip installing one thing is a huge problem for a lot of people, but the alternative is harder.
BiteCode_dev 3y ago
That's more of cultural problem in the Python community. If I provide an end user software to my client written in Python (so not a backend, not a lib...), I will compile it with nuitka (https://github.com/Nuitka/Nuitka https://github.com/Nuitka/Nuitka) and hide the stack trace (https://www.bitecode.dev/p/why-and-how-to-hide-the-python-stack https://www.bitecode.dev/p/why-and-how-to-hide-the-python-st...) to provide a stand alone executable. This means the users don't have to know it's made with Python or install anything, and it just works. However, Python is not like Go or Rust, and providing such an installer requires more work, so a huge part of the user base (which have a lot of non professional coders) don't have the skill, time or resources to do it. And few people make the promotion of it. I should write an article on that because really, nobody wants to setup python just to use a tool. Do check out nuitka though, it's has great support for QT, numpy, advanced Python constructs and has a permissive learning curve. You may even get a up to a X4 perf boot. Not cross compilation though. There is a startup idea right there.
- pbronez 3y ago
  Pipx helps with this use case.. though you still need Pipx!
  - playingalong 3y ago
    Honestly I cannot tell this irony or not. 14 standards? We need one more
    - fragmede 3y ago
      http://xkcd.com/927 http://xkcd.com/927
  - BiteCode_dev 3y ago
    pipx is just displacing problems: - you have to install pipx and they recommand to install it... with pip. Back to all the same problems. - pipx implicitly use a python version underneath. For things like mypy or black, you get stuck to the syntax of this version, and you'll get errors a lot of users can't solve - pipx is not stable enough, it just fails sometimes - people have to know about a third party tool - it adds to the paradox of choice - pipx PATH injection is hit or miss I should have listed pipx in the article, it's not a solution, it's a crutch.
Alifatisk 3y ago
> When I see something that I'd like to install, and the docs say "written in Python", my heart sinks. Why is this so relatable? I have almost only had bad experience with projects written in Python, it's so clunky to get started with. The only proper "solution" to deal with this is shipping the project in a container so no user have to deal with it again.
- perrygeo 3y ago
  The rise of Docker and the rise of Python as the most popular application programming language is not a coincidence.
eurekin 3y ago
Sample apps using PyTorch are best examples of an adventure, when ran on Windows. One needs to carefuly consider a WSL2 machine, full vm or Anaconda, because each comes with some caveats. WSL doesn't implement sound/microphone access, full vm's don't pass through the main display GPU, Anaconda based ones complain that the host system doesn't support hard symlinks (unless the Developer mode is turned on)
the__alchemist 3y ago
> I don't normally use Python. When I see something that I'd like to install, and the docs say "written in Python", my heart sinks. I do use python regularly, but still have this reaction. If I've set up a project that runs a CLI script, or Django web server etc, I can manage dependencies. If it's someone else's code, all hope is lost! It also makes me wonder why it's not wrapped into a binary using one of the various tools available.
- dagw 3y ago
  It also makes me wonder why it's not wrapped into a binary using one of the various tools available. Because they're all hard to use, fall down in all kinds of corner cases, tend to produce huge binaries and have really bad startup time.
Ingaz 3y ago
There a lot of way to produce self-contained binaries but .. there are a lot of buts. My hopes in PyOxider path. Look at Anki SRS build
bjourne 3y ago
Maybe you are doing something wrong? I have installed countless Python packages using "pip install --user pkg-name" If something fucks up i just rm -rf ~/.local/lib/python3.11/site-packages and reinstall the package. I've stuck to this workflow for over ten years and the only major issue I have had with it was with TensorFlow which didn't support the latest Python version. Nowadays, I have to add the flag "--break-system-packages" but the workflow still works as well as it ever did.
globular-toast 3y ago
This doesn't match my experience at all. Can you give any concrete examples of packages that are difficult to install because they are written in Python?
- jononor 3y ago
  "difficult" is in the eyes of the beholder. But one quite simple and well maintained package, which many have still have problems with is librosa. See for example Stack Overflow questions about it or the issue tracker. Some of the challenges come from having dependencies on native libraries (audio codecs), that need to be installed manually. Others come from using numba, which break on every new Python version.
  - globular-toast 3y ago
    librosa is a library. That means you need to learn to do Python development to be able to use it obviously. This is the problem. People aren't even talking about the same thing. This thread started off as something about single file distributables and the only concrete example given is a library.
joppy 3y ago
Use pipx. It works well.

by jwr 3y ago

bombolo 3y ago

For me if I can do apt install is good. No care which language it's written in.

rgoulter 3y ago
Sure, it's convenient if there's already a system package; in which case, yes, the language is irrelevant. However, if you want to run software which isn't available as a system package, then you'll need to know how to build/run the software.
- roflyear 3y ago
  Isn't that true of many languages?
  - arp242 3y ago
    Kind of, but some languages are definitely easier to build than others (on average).
  - rgoulter 3y ago
    Surely "to run the program from source, you need to know how to build/run it" applies to every language. But I'm more comfortable working through problems I might encounter in languages I've worked with, than in languages I'm not familiar with. I'd say Python is notorious, since you're probably going to hear about at least six or seven ways people are able to run things; so if you're not familiar with it, you're likely to do something wrong. -- e.g. elsewhere in this thread, you've pointed out that stuff should be run from a venv; that's something you have to know.
    - roflyear 3y ago
      It's true: some people like Docker for Python, some like venv, some like conda... some lose it entirely and install everything locally! (acting like I haven't done this at least once!)
  - jeltz 3y ago
    Yes, but Python is among the harder languages to build while other languages can be a breeze. And that is despite having used Python much more than some of them.
    - bombolo 3y ago
      Python is rather easy. python3 -m venv bla; . bla/bin/activate; pip install -r requirements.txt; ./command Of course badly written code is buggy and will require bug fixing because it only works on the machine of the developer. I'm not sure this isn't true in other languages. I've seen go CLI utilities that were dynamically linking Gnome libraries and could not start without installing Gnome.
  - dagw 3y ago
    Many languages have much better tools that make it easy to compile/bundle an artefact that can be run without the end user having to build or install anything.
    - bombolo 3y ago
      Much better tools? Welcome to use C++ without a linux distribution providing compiler and headers!
BiteCode_dev 3y ago
Yes, if you are a user. But if you are a developer, it's not great.

theshrike79 3y ago

This was the main reason I started writing my random tools and scripts in Go. I can just write, cross-compile and deploy. Trying to get a "simple" Python tool with library dependencies working on a random server is a huge pain in the keister I'd rather not deal with if I have the option. This became even easier now that I can "simply" feed my old Python stuff to GPT4 and have it transform it to mostly-working Go code =)

eurekin 3y ago
Interesting. Where are those tools deployed to? Is there any runtime needed on the client? How can I grab a go tool on a pristine machine?
- jbarrs 3y ago
  Not the original commenter, but Go programs are all statically-compiled, with everything needed at run time bundled into the executable. Go does use a runtime, but that's incorporated into the exe, so no external libraries required. As such, you can get a program written in Go running on a new machine just by downloading it :) For machines where you may actually have a Go installation already, the majority of programs now can also be installed from source via Go's built-in package manager, which installs it to your home directory. A lot of dev utils, like go-imports, the language server, etc. are typically installed this way. It's usually just a single command (though the exact command can vary between projects, but is usually signposted fairly clearly).
- theshrike79 3y ago
  For 99% of stuff I need to move either one statically compiled Go binary and maybe a configuration yaml. That's it. With Python I need to have Python installed in there, and it has to be the correct version because of <reasons>. Then I need to figure out how I can install the dependencies for the program, with the correct version(s). This usually would require me to figure out what's the current virtual env du jour and how do I get it working on a random *nix distro on the target computer. Then I can install what's in requirements.txt or whatever the venv tool is using. And now I can maybe run it if all went well. And when I need to update said program, with Go I can just scp/rsync a new binary on top of the old one at it'll Just Work. With Python I need to move the Python source file(s) there, check for possible updates to libraries, update them using the venv tool and re-check that I have the correct Python version installed once more. Now picture a situation where I need to do the process above for, let's say 12 different servers on different clients after every fortnight. All servers are Intel-based, but run a variety of Linux distros. I can use the exact same Go binary for every single one, the Python solution is a complete clusterf... I do need to spend a bit more time when developing a Go solution compared to Python, but the ease of deployment is definitely worth it.

throwawaaarrgh 3y ago

Most programmers don't give a crap about the user. Python assumes that the user will be a Python programmer, and want to maintain a whole independent part of their system just to run a Python program.

jrib 3y ago

What do you normally use? I'm curious what context you are comparing with.

jeltz 3y ago
Perl, Ruby, JavaScript, PHP, Bash, Go, C, C++, Rust and even Java is easier to deal with than Python tools. Seeing that it is a Python tool usually make my heart drop a lot and makes me look for alternatives before attempting to get it running.
wongarsu 3y ago
I assume the gold standard is just downloading a compiled binary and using that. Which is the norm for almost everyone ... except python projects. When it comes to compiling/running from source, Python is in the middle of the pack. It's not as smooth as rust (install cargo from your package manager or from rustup, run "cargo build") but not as bad as C++ (run through the install section of the readme of each dependency, then debug whatever build system the project is using).

supriyo-biswas 3y ago

The lack of a way to produce a self contained binary is why I switched away from Python for a personal project of mine. However, to address the other part of your comment, pipx works quite well for installing command line applications.

globular-toast 3y ago
Why does it matter if a program is a self-contained binary? It seems like such an odd requirement. If you really want that it seems in principle easy to compile it all into one script with no imports and put a shebang at the top. What's the point, though? Pyinstaller is pretty good for if you need a standalone installer.
- spicyusername 3y ago
  Self contained binaries are very easy to distribute and use.
- bheadmaster 3y ago
  > Why does it matter if a program is a self-contained binary? Because simple is better than complex. I'd rather download a binary than download a binary + dependencies + set them up + bookkeep for when I want to delete all those files. > Pyinstaller is pretty good for if you need a standalone installer. PyInstaller doesn't work for many edge cases. If you're using a Python package that uses a compiled binary written in another language, good luck on your way down the rabbit hole of PyInstaller config. I personally could not succeed in packaging a uvicorn app, for example.
  - redsaber 3y ago
    > I personally could not succeed in packaging a uvicorn app, for example. I kinda did managed to run a fastapi with uvicorn and pyinstaller. And I did nothing special
    - bheadmaster 3y ago

roflyear 3y ago

pip installing one thing is usually not a problem. You can do the binary thing with Python for sure. Does Lisp really work much better? Part of the issue with Python is versions of the language. So you really shouldn't be installing packages locally in most cases, but you should use one of these tools so you're running your dev environment isolated from other environments.

crabbone 3y ago
> Does Lisp really work much better? Most of the popular ones are a lot better than Python. Python might be comparable to CAD Lisp or that abomination embedded in GIMP (though I think they stopped supporting it) in terms of quality of design and execution. > So you really shouldn't be installing packages locally in most cases Where do you suggest I install them instead? On someone else's computer? What kind of advise is that?..
BiteCode_dev 3y ago
If you read the article, it argues that yes, pip installing one thing is a huge problem for a lot of people, but the alternative is harder.

BiteCode_dev 3y ago

That's more of cultural problem in the Python community. If I provide an end user software to my client written in Python (so not a backend, not a lib...), I will compile it with nuitka (https://github.com/Nuitka/Nuitka https://github.com/Nuitka/Nuitka) and hide the stack trace (https://www.bitecode.dev/p/why-and-how-to-hide-the-python-stack https://www.bitecode.dev/p/why-and-how-to-hide-the-python-st...) to provide a stand alone executable. This means the users don't have to know it's made with Python or install anything, and it just works. However, Python is not like Go or Rust, and providing such an installer requires more work, so a huge part of the user base (which have a lot of non professional coders) don't have the skill, time or resources to do it. And few people make the promotion of it. I should write an article on that because really, nobody wants to setup python just to use a tool. Do check out nuitka though, it's has great support for QT, numpy, advanced Python constructs and has a permissive learning curve. You may even get a up to a X4 perf boot. Not cross compilation though. There is a startup idea right there.

pbronez 3y ago
Pipx helps with this use case.. though you still need Pipx!
- playingalong 3y ago
  Honestly I cannot tell this irony or not. 14 standards? We need one more
  - fragmede 3y ago
    http://xkcd.com/927 http://xkcd.com/927
- BiteCode_dev 3y ago
  pipx is just displacing problems: - you have to install pipx and they recommand to install it... with pip. Back to all the same problems. - pipx implicitly use a python version underneath. For things like mypy or black, you get stuck to the syntax of this version, and you'll get errors a lot of users can't solve - pipx is not stable enough, it just fails sometimes - people have to know about a third party tool - it adds to the paradox of choice - pipx PATH injection is hit or miss I should have listed pipx in the article, it's not a solution, it's a crutch.

Alifatisk 3y ago

> When I see something that I'd like to install, and the docs say "written in Python", my heart sinks. Why is this so relatable? I have almost only had bad experience with projects written in Python, it's so clunky to get started with. The only proper "solution" to deal with this is shipping the project in a container so no user have to deal with it again.

perrygeo 3y ago
The rise of Docker and the rise of Python as the most popular application programming language is not a coincidence.

eurekin 3y ago

Sample apps using PyTorch are best examples of an adventure, when ran on Windows. One needs to carefuly consider a WSL2 machine, full vm or Anaconda, because each comes with some caveats. WSL doesn't implement sound/microphone access, full vm's don't pass through the main display GPU, Anaconda based ones complain that the host system doesn't support hard symlinks (unless the Developer mode is turned on)

the__alchemist 3y ago

> I don't normally use Python. When I see something that I'd like to install, and the docs say "written in Python", my heart sinks. I do use python regularly, but still have this reaction. If I've set up a project that runs a CLI script, or Django web server etc, I can manage dependencies. If it's someone else's code, all hope is lost! It also makes me wonder why it's not wrapped into a binary using one of the various tools available.

dagw 3y ago
It also makes me wonder why it's not wrapped into a binary using one of the various tools available. Because they're all hard to use, fall down in all kinds of corner cases, tend to produce huge binaries and have really bad startup time.

Ingaz 3y ago

There a lot of way to produce self-contained binaries but .. there are a lot of buts. My hopes in PyOxider path. Look at Anki SRS build

bjourne 3y ago

Maybe you are doing something wrong? I have installed countless Python packages using "pip install --user pkg-name" If something fucks up i just rm -rf ~/.local/lib/python3.11/site-packages and reinstall the package. I've stuck to this workflow for over ten years and the only major issue I have had with it was with TensorFlow which didn't support the latest Python version. Nowadays, I have to add the flag "--break-system-packages" but the workflow still works as well as it ever did.

globular-toast 3y ago

This doesn't match my experience at all. Can you give any concrete examples of packages that are difficult to install because they are written in Python?

jononor 3y ago
"difficult" is in the eyes of the beholder. But one quite simple and well maintained package, which many have still have problems with is librosa. See for example Stack Overflow questions about it or the issue tracker. Some of the challenges come from having dependencies on native libraries (audio codecs), that need to be installed manually. Others come from using numba, which break on every new Python version.
- globular-toast 3y ago
  librosa is a library. That means you need to learn to do Python development to be able to use it obviously. This is the problem. People aren't even talking about the same thing. This thread started off as something about single file distributables and the only concrete example given is a library.

joppy 3y ago

Use pipx. It works well.