4 ms·

I'm confronted with a similar problem frequently. I have a growing bash script and it's slowly growing in complexity. Once bash scripts become sufficiently long

I'm confronted with a similar problem frequently. I have a growing bash script and it's slowly growing in complexity. Once bash scripts become sufficiently long, I find editing them later to be very annoying. So instead, at some point I change the language entirely and write a utility in python/lua/c/whatever other language I want. As time goes on, my limit for "sufficient complexity" to justify leaving bash and using something like python has dropped radically. Now I follow the rule that as soon as I do something "nontrivial", it should be in a scripting language. As a side-effect, my bash scripting skills are worse than they once were. And now the scope of what I consider "trivial" is shrinking!

AtlasBarfed 2y ago
Isn't this perfect for LLM? You know, assuming they transpile well, I haven't tried a solid one yet. I wonder if kernel code rewrites in rust with Llama (obviously reviewed are up to snuff.
maccard 2y ago
I agree. My limit is pretty much one you start branching or looping, it should be in another tool. If that seems low to you, that’s the point
fieu 2y ago
I have exactly the same issue. I maintain a project called discord.sh which sends Discord webhooks via pure Bash (and a little bit of jq and curl). At some point I might switch over to Go or C. https://github.com/fieu/discord.sh https://github.com/fieu/discord.sh
- wiether 2y ago
  First of all, thank you for your work! I'm using it daily for many years now and it does exactly what I expect it to do. Now I'm a little concerned by the end of your message because it could make its usage a bit trickier... My main usecase is to curl the raw discord.sh file from GitHub in a Dockerfile and put in in /user/local/bin, so then I can _discord.sh_ anytime I need it. Mostly used for CI images. The only constraint is to install jq if it's not already installed on the base image. Switching to Go or C would make the setup much harder I'm afraid
  - Imustaskforhelp 2y ago
    I suppose https://www.youtube.com/watch?v=eRHlFkomZJg https://www.youtube.com/watch?v=eRHlFkomZJg If you don't want to watch video , then I can link the tool it uses https://github.com/igor-petruk/scriptisto/wiki/Writing-scripts#builds-in-docker https://github.com/igor-petruk/scriptisto/wiki/Writing-scrip...
  - Imustaskforhelp 2y ago
    From what it seems , it seems that its possible to run this thing without installing go,rust,c itself to quote from the page With scriptisto you can build your binary in an automatically managed Docker container, without having compilers installed on host. If you build your binary statically, you will be able to run it on host. There are a lot if images that help you building static binaries, starting from alpine offering a MUSL toolchain, to more specialized images. Find some docker-* templates via scriptisto new command. Examples: C, Rust. No need to have anything but Docker installed! Builds in Docker enabled by populating the docker_build config entry, defined as such: Also I am watching the video again because I had viewed it a looong time ago !
  - benediktwerner 2y ago
    Why would that make the setup harder? If they provide a statically-linked executable, you can just download and run it, without even the need to install jq or anything else. It's not like they'd provide Go code and ask you to compile it yourself. Go isn't Python.
    - xenophonf 2y ago
      Even Python isn't Python. https://pyinstaller.org/ https://pyinstaller.org/ Works great. I use this instead of pipx.
  - fieu 2y ago
    Thank you for using the project! On the concern of it would be harder to setup, I think it would be easier in fact, you would simply curl the Go or C statically generated binary to your path and would alleviate the need for jq or curl to be installed alongside. I think the reason I haven’t made the switch yet is I like Bash (even though my script is getting pretty big), and in a way it’s a testament to what’s possible in the language. Projects like https://github.com/acmesh-official/acme.sh https://github.com/acmesh-official/acme.sh really show the power of Bash. That and I think the project would need a name change, and discord.sh as a name gets the point across better than anything I can think of.
ComputerGuru 2y ago
My problem with python is startup time, packaging complexity (either dependency hell or full blown venv with pipx/uv). I’ve been rewriting shell scripts to either Makefiles (crazy but it works and is rigorous and you get free parallelism) or rust “scripts” [0] depending on their nature (number of outputs, number of command executions, etc) Also, using a better shell language can be a huge productivity (and maintenance and sanity) boon, making it much less “write once, read never”. Here’s a repo where I have a mix of fish-shell scripts with some converted to rust scripts [1]. [0]: https://neosmart.net/blog/self-compiling-rust-code/ https://neosmart.net/blog/self-compiling-rust-code/ [1]: https://github.com/mqudsi/ffutils https://github.com/mqudsi/ffutils
- roelschroeven 2y ago
  I've often read that people have a problem with Python's startup time, but that's not at all my experience. Yes, if you're going to import numpy or pandas or other heavy packages, that can be annoyingly slow. But we're talking using Python as a bash script alternative here. That means (at least to me) importing things like subprocess, pathlib. In my experience, that doesn't take long to start. $ cat helloworld.py #!/usr/bin/env python3 import subprocess from pathlib import Path print("Hello, world!\n") $ time ./helloworld.py Hello, world! real 0m0.034s user 0m0.016s sys 0m0.016s 34 milliseconds doesn't seem a lot of time to me. If you're going to run it in a tight loop than yes, that's going to be annoying, but in interactive use I don't even notice delays as small as that. As for packaging complexity: when using Python as a bash script alternative, I mostly can easily get by with using only stuff from the standard library. In that case, packaging is trivial. If I do need other packages then yes, that can be major nuisance.
  - drdrey 2y ago
    once you start importing more packages, you easily end up with 100+ ms startup time
    - robertlagrant 2y ago
      At that point you're far beyond a bash script alternative though, aren't you?
NoMoreNicksLeft 2y ago
Yesterday, I had a problem where wget alone could do 98% of what I wanted. I could restrict which links it followed, but the files I needed to retrieve were a url parameter passed in with a header redirect at the end. I spent an hour relearning all the obscure stuff in wget to get that far. The python script is 29 lines, and it turns out I can just target a url that responds with json and dig the final links out of that. Usually though, yeh, everything starts as a bash script.
bigstrat2003 2y ago
I definitely agree. Bash is such an unpleasant language to work with, with so many footguns, that I reach for a language like Python as soon as I'm beyond 10 lines or so.

AtlasBarfed 2y ago

Isn't this perfect for LLM? You know, assuming they transpile well, I haven't tried a solid one yet. I wonder if kernel code rewrites in rust with Llama (obviously reviewed are up to snuff.

maccard 2y ago

I agree. My limit is pretty much one you start branching or looping, it should be in another tool. If that seems low to you, that’s the point

fieu 2y ago

I have exactly the same issue. I maintain a project called discord.sh which sends Discord webhooks via pure Bash (and a little bit of jq and curl). At some point I might switch over to Go or C. https://github.com/fieu/discord.sh https://github.com/fieu/discord.sh

wiether 2y ago
First of all, thank you for your work! I'm using it daily for many years now and it does exactly what I expect it to do. Now I'm a little concerned by the end of your message because it could make its usage a bit trickier... My main usecase is to curl the raw discord.sh file from GitHub in a Dockerfile and put in in /user/local/bin, so then I can _discord.sh_ anytime I need it. Mostly used for CI images. The only constraint is to install jq if it's not already installed on the base image. Switching to Go or C would make the setup much harder I'm afraid
- Imustaskforhelp 2y ago
  I suppose https://www.youtube.com/watch?v=eRHlFkomZJg https://www.youtube.com/watch?v=eRHlFkomZJg If you don't want to watch video , then I can link the tool it uses https://github.com/igor-petruk/scriptisto/wiki/Writing-scripts#builds-in-docker https://github.com/igor-petruk/scriptisto/wiki/Writing-scrip...
- Imustaskforhelp 2y ago
  From what it seems , it seems that its possible to run this thing without installing go,rust,c itself to quote from the page With scriptisto you can build your binary in an automatically managed Docker container, without having compilers installed on host. If you build your binary statically, you will be able to run it on host. There are a lot if images that help you building static binaries, starting from alpine offering a MUSL toolchain, to more specialized images. Find some docker-* templates via scriptisto new command. Examples: C, Rust. No need to have anything but Docker installed! Builds in Docker enabled by populating the docker_build config entry, defined as such: Also I am watching the video again because I had viewed it a looong time ago !
- benediktwerner 2y ago
  Why would that make the setup harder? If they provide a statically-linked executable, you can just download and run it, without even the need to install jq or anything else. It's not like they'd provide Go code and ask you to compile it yourself. Go isn't Python.
  - xenophonf 2y ago
    Even Python isn't Python. https://pyinstaller.org/ https://pyinstaller.org/ Works great. I use this instead of pipx.
- fieu 2y ago
  Thank you for using the project! On the concern of it would be harder to setup, I think it would be easier in fact, you would simply curl the Go or C statically generated binary to your path and would alleviate the need for jq or curl to be installed alongside. I think the reason I haven’t made the switch yet is I like Bash (even though my script is getting pretty big), and in a way it’s a testament to what’s possible in the language. Projects like https://github.com/acmesh-official/acme.sh https://github.com/acmesh-official/acme.sh really show the power of Bash. That and I think the project would need a name change, and discord.sh as a name gets the point across better than anything I can think of.

ComputerGuru 2y ago

My problem with python is startup time, packaging complexity (either dependency hell or full blown venv with pipx/uv). I’ve been rewriting shell scripts to either Makefiles (crazy but it works and is rigorous and you get free parallelism) or rust “scripts” [0] depending on their nature (number of outputs, number of command executions, etc) Also, using a better shell language can be a huge productivity (and maintenance and sanity) boon, making it much less “write once, read never”. Here’s a repo where I have a mix of fish-shell scripts with some converted to rust scripts [1]. [0]: https://neosmart.net/blog/self-compiling-rust-code/ https://neosmart.net/blog/self-compiling-rust-code/ [1]: https://github.com/mqudsi/ffutils https://github.com/mqudsi/ffutils

roelschroeven 2y ago
I've often read that people have a problem with Python's startup time, but that's not at all my experience. Yes, if you're going to import numpy or pandas or other heavy packages, that can be annoyingly slow. But we're talking using Python as a bash script alternative here. That means (at least to me) importing things like subprocess, pathlib. In my experience, that doesn't take long to start. $ cat helloworld.py #!/usr/bin/env python3 import subprocess from pathlib import Path print("Hello, world!\n") $ time ./helloworld.py Hello, world! real 0m0.034s user 0m0.016s sys 0m0.016s 34 milliseconds doesn't seem a lot of time to me. If you're going to run it in a tight loop than yes, that's going to be annoying, but in interactive use I don't even notice delays as small as that. As for packaging complexity: when using Python as a bash script alternative, I mostly can easily get by with using only stuff from the standard library. In that case, packaging is trivial. If I do need other packages then yes, that can be major nuisance.
- drdrey 2y ago
  once you start importing more packages, you easily end up with 100+ ms startup time
  - robertlagrant 2y ago
    At that point you're far beyond a bash script alternative though, aren't you?

NoMoreNicksLeft 2y ago

Yesterday, I had a problem where wget alone could do 98% of what I wanted. I could restrict which links it followed, but the files I needed to retrieve were a url parameter passed in with a header redirect at the end. I spent an hour relearning all the obscure stuff in wget to get that far. The python script is 29 lines, and it turns out I can just target a url that responds with json and dig the final links out of that. Usually though, yeh, everything starts as a bash script.

bigstrat2003 2y ago

I definitely agree. Bash is such an unpleasant language to work with, with so many footguns, that I reach for a language like Python as soon as I'm beyond 10 lines or so.