4 ms·

Depending on your job description it is completely fine to not write shell scripts. For anything bigger than a screenful of lines or something that would need

by bhaak 3y ago

Depending on your job description it is completely fine to not write shell scripts. For anything bigger than a screenful of lines or something that would need proper unit testing or something that other people who are likely to be less versed in shell scripting need to maintain, think twice about using shell scripts. Unixoid machines can use anything as scripts and using a language that breathes the programming language designs of the 1970s is not the best choice.

teddyh 3y ago
> or something that would proper unit testing <https://github.com/kward/shunit2 https://github.com/kward/shunit2>
- nickcox 3y ago
  And several others: https://github.com/dodie/testing-in-bash https://github.com/dodie/testing-in-bash
softwaredoug 3y ago
I typically agree, but I don't think its as simple as a "screenful of lines" criteria. I find it useful when most of your work is wiring other command line tools together. When that's most of your work, shell scripting feels like it has the best affordances. Anything that gets too fancy beyond this, with anything that resembles an algorithm or computation, you'll feel real pain :) Bash doesn't even support floating point!
- nextaccountic 3y ago
  > Bash doesn't even support floating point! But bc (and dc) does, and shell authors deemed this was good enough. And that's part of the problem with the "Unix philosophy"
  - softwaredoug 3y ago
    Yeah every invocation of bc is a call to a command. So if you do it in a loop, its going to be very slow. I had to do this once. It was so slow, I decided to use integers in kind of fixed-point math where I just multiply all the floats by 100 and work with ints. zsh supports float though
    - worksonmine 3y ago
      Did you consider preparing the arguments in the loop then run them all at once?
- hereonout2 3y ago
  For me this is is it, if you're stringing together a load of other command line tools then a large well structured shell script is perfectly fine. A typical task in my day job is extracting metadata from and transcoding large AV archives, I always use a bash for this because I'm mainly stringing together tools like ffmpeg or sox, making directories, tempfiles, creating chesksum manifests, etc. Also I typically do this on large CPU machines with many cores, I can parallelize operations in one line using xargs and a bash function. Doing stuff like this in python is actually a real pain in comparison.
  - nerdponx 3y ago
    > python There's a library for Node called Zx (https://github.com/google/zx https://github.com/google/zx) that provides a more shell-liked interface for Node. I've been working on something in Python, inspired by Zx and Xonsh (https://xon.sh/ https://xon.sh/), with syntax like this: from pathlib import Path from pyshell import Pipeline args = ["123", "456"] f = Path("out.txt") with Pipeline() as p: p.exec("a") | p.exec("b", *args) | p.exec("c", "x", "y") >> f Personally, the above would be hugely useful to me in reducing my temptation to write shell scripts, and significantly reducing the annoyance of doing these things in Python. Is this something you'd use? It's slow going for me because it basically involves writing a compiler and I have approximately zero idea of what I'm doing. Knowing that other people would find it useful helps me stay motivated to keep working on it! Or if someone who theoretically is the primary audience doesn't find it useful, it might mean that I should spend my energy on other things.
    - hereonout2 3y ago
      This looks cool, especially if one needed to do a number of pipe operations from python, I'm not sure of an elegant way to do that already. Honestly I might be the wrong audience though, I've spent many years shell scripting and I find the flexibility of using the command line tools so great I choose to use it first despite the inadequacies of bash. I wonder if users with my point of view would ever be tempted away. Also the shell is an interactive repl, which to me is where a lot of the power lies. The number of times I've asked a junior things like what the total duration of a set of audio files is, or the average length or something. They often come back with a python script or worse a jupyter notebook and pandas. The answer can also often be found by combining things like grep and awk in a few seconds once you've become acquainted with the tools, I guess I do try and demonstrate / preach this where I can.
- chasil 3y ago
  If you want floating point, run ksh93. If you prefer rigid adherence to standards, use awk.
- JNRowe 3y ago
  There was a really awful and amazing floating point implementation posted here¹ a couple of years ago, with a few interesting comments too². I remember it in part because I was surprised there wasn't more discussion. ¹ https://github.com/clarity20/shellmath https://github.com/clarity20/shellmath ² https://news.ycombinator.com/item?id=26250743 https://news.ycombinator.com/item?id=26250743
amelius 3y ago
Yes. Can someone please write an LLM that can translate shell scripts to Python, for those cases when my shell script starts exceeding a screenful of lines.
- jpc0 3y ago
  To be fair for most of the shell scripts I write it would be more work to put it into an LLM and then verify what it output is correct than just retyping it in python. I tend towards python these days if I need more than 1 variable since at that point I'm probably in the weeds already. If it's just a ton of commands in succession though, I leave it bash. I have a few scripts in bash and PowerShell for weird ffmpeg operations I generally don't want to remember, stuff like loudness normalisation and converting to specific formats with specific parameters using ffmpeg.

by bhaak 3y ago

teddyh 3y ago

> or something that would proper unit testing <https://github.com/kward/shunit2 https://github.com/kward/shunit2>

nickcox 3y ago
And several others: https://github.com/dodie/testing-in-bash https://github.com/dodie/testing-in-bash

softwaredoug 3y ago

I typically agree, but I don't think its as simple as a "screenful of lines" criteria. I find it useful when most of your work is wiring other command line tools together. When that's most of your work, shell scripting feels like it has the best affordances. Anything that gets too fancy beyond this, with anything that resembles an algorithm or computation, you'll feel real pain :) Bash doesn't even support floating point!

nextaccountic 3y ago
> Bash doesn't even support floating point! But bc (and dc) does, and shell authors deemed this was good enough. And that's part of the problem with the "Unix philosophy"
- softwaredoug 3y ago
  Yeah every invocation of bc is a call to a command. So if you do it in a loop, its going to be very slow. I had to do this once. It was so slow, I decided to use integers in kind of fixed-point math where I just multiply all the floats by 100 and work with ints. zsh supports float though
  - worksonmine 3y ago
    Did you consider preparing the arguments in the loop then run them all at once?
hereonout2 3y ago
For me this is is it, if you're stringing together a load of other command line tools then a large well structured shell script is perfectly fine. A typical task in my day job is extracting metadata from and transcoding large AV archives, I always use a bash for this because I'm mainly stringing together tools like ffmpeg or sox, making directories, tempfiles, creating chesksum manifests, etc. Also I typically do this on large CPU machines with many cores, I can parallelize operations in one line using xargs and a bash function. Doing stuff like this in python is actually a real pain in comparison.
- nerdponx 3y ago
  > python There's a library for Node called Zx (https://github.com/google/zx https://github.com/google/zx) that provides a more shell-liked interface for Node. I've been working on something in Python, inspired by Zx and Xonsh (https://xon.sh/ https://xon.sh/), with syntax like this: from pathlib import Path from pyshell import Pipeline args = ["123", "456"] f = Path("out.txt") with Pipeline() as p: p.exec("a") | p.exec("b", *args) | p.exec("c", "x", "y") >> f Personally, the above would be hugely useful to me in reducing my temptation to write shell scripts, and significantly reducing the annoyance of doing these things in Python. Is this something you'd use? It's slow going for me because it basically involves writing a compiler and I have approximately zero idea of what I'm doing. Knowing that other people would find it useful helps me stay motivated to keep working on it! Or if someone who theoretically is the primary audience doesn't find it useful, it might mean that I should spend my energy on other things.
  - hereonout2 3y ago
    This looks cool, especially if one needed to do a number of pipe operations from python, I'm not sure of an elegant way to do that already. Honestly I might be the wrong audience though, I've spent many years shell scripting and I find the flexibility of using the command line tools so great I choose to use it first despite the inadequacies of bash. I wonder if users with my point of view would ever be tempted away. Also the shell is an interactive repl, which to me is where a lot of the power lies. The number of times I've asked a junior things like what the total duration of a set of audio files is, or the average length or something. They often come back with a python script or worse a jupyter notebook and pandas. The answer can also often be found by combining things like grep and awk in a few seconds once you've become acquainted with the tools, I guess I do try and demonstrate / preach this where I can.
chasil 3y ago
If you want floating point, run ksh93. If you prefer rigid adherence to standards, use awk.
JNRowe 3y ago
There was a really awful and amazing floating point implementation posted here¹ a couple of years ago, with a few interesting comments too². I remember it in part because I was surprised there wasn't more discussion. ¹ https://github.com/clarity20/shellmath https://github.com/clarity20/shellmath ² https://news.ycombinator.com/item?id=26250743 https://news.ycombinator.com/item?id=26250743

amelius 3y ago

Yes. Can someone please write an LLM that can translate shell scripts to Python, for those cases when my shell script starts exceeding a screenful of lines.

jpc0 3y ago
To be fair for most of the shell scripts I write it would be more work to put it into an LLM and then verify what it output is correct than just retyping it in python. I tend towards python these days if I need more than 1 variable since at that point I'm probably in the weeds already. If it's just a ton of commands in succession though, I leave it bash. I have a few scripts in bash and PowerShell for weird ffmpeg operations I generally don't want to remember, stuff like loudness normalisation and converting to specific formats with specific parameters using ffmpeg.