3 ms·

> Packaging, dependency management, and reproducibility in shell land are still stuck in the Stone Ages. IMO it should stay that way, because any script that n

by bigstrat2003 1y ago

> Packaging, dependency management, and reproducibility in shell land are still stuck in the Stone Ages. IMO it should stay that way, because any script that needs those things is way past the point where shell is a reasonable choice. Shell scripts should be small, 20 lines or so. The language just plain sucks too much to make it worth using for anything bigger.

pxc 1y ago
When you solve the dependency management issue for shell scripts, you can also use newer language features because you can ship a newer interpreter the same way you ship whatever external dependencies you have. You don't have to limit yourself to what is POSIX, etc. Depending on how you solve it, you may even be able to switch to a newer shell with a nicer language. (And doing so may solve it for you; since PowerShell, newer shells often come with a dependency management layer.) > any script that needs those things It's not really a matter of needing those things, necessarily. Once you have them, you're welcome to write scripts in a cleaner, more convenient way. For instance, all of my shell scripts used by colleagues at work just use GNU coreutils regardless of what platform they're on. Instead of worrying about differences in how sed behaves with certain flags, on different platforms, I simply write everything for GNU sed and it Just Works™. Do those scripts need such a thing? Not necessarily. Is it nicer to write free of constraints like that? Yes! Same thing for just choosing commands with nicer interfaces, or more unified syntax... Use p7zip for handling all your archives so there's only one interface to think about. Make heavy use of `jq` (a great language) for dealing with structured data. Don't worry about reading input from a file and then writing back to it in the same pipeline; just throw in `sponge` from moreutils. > The language just plain sucks too much There really isn't anything better for invoking external programs. Everything else is way clunkier. Maybe that's okay, but when I've rewritten large-ish shell scripts in other languages, I often found myself annoyed with the new language. What used to be a 20-line shell script can easily end up being 400 lines in a "real" language. I kind of agree with you, of course. POSIX-ish shells have too much syntax and at the same time not enough power. But what I really want is a better shell language, not to use some interpreted non-shell language in their place.
- MatmaRex 1y ago
  Broke: Dependency management used for shell scripts Woke: Dependency management used for installing an interpreter for a better programming language to write your script in it Bespoke: Dependency management used for installing your script
- m2f2 1y ago
  Nice, if only you could count on having it installed on your fleet, and your fleet is 100pct Linux, no AIX, no HPUX, no SOLARIS, no SUSE on IBM Power.... Been there, tried to, got a huge slap in the face.
  - kstrauser 1y ago
    Been there, done that. I am so glad I don’t have to deal with all that insanity anymore. In the build farm I was responsible for, I was always happy to work on the Linux and BSD boxes. AIX and HPUX made me want to throw things. At least the Itanium junk acted like a normal server, just a painfully slow one. I will never voluntarily run a bunch of non-Linux/BSD servers again.
    - lenerdenator 1y ago
      I honestly don't get why there are still a bunch of non-Linux/BSD servers, at least if the goal is to do UNIX-y stuff. I haven't touched AIX or HPUX in probably a decade and I thought they were a weird idea back then: proprietary UNIX? Is it still 1993?
      - kstrauser 1y ago
        At the time (10 years ago) I worked for a company with enormous customers who had all kinds of different deployment targets. I bet that list is a lot shorter today. I hope so, for their sake. shudder
xavdid 1y ago
My rule of thumb is that as soon as I write a conditional, it's time to upgrade bash to Python/Node/etc. I shouldn't have to search for the nuances of `if` statements every time I need to write them.
- w0m 1y ago
  Historically; my rule of thumb is as soon as I can't see the ~entire script without scrolling - time to rewrite in Python/ansible. I Think about the rewrite, but it usually takes awhile to do it (if ever)
- pxc 1y ago
  What nuances are there to if statements, exactly? An if statement in, for instance bash, just runs any command and then runs one of two blocks of code based on the exit status of that command. If the exit status is truthy, it runs what follows the `then`. If it's falsey, it rhns what follows the `else`. (`elsif` is admittedly gratuitous syntax— it would be better if it were just implemented as an if inside an else statement.) This seems quite similar to other programming languages and like not very much to remember. I'll admit that one thing I do in my shell scripts is avoid "fake syntax"— I never use `[` or `[[` because these obscure the real structure of the statements for the sake of cuteness. I just write `test`, which makes clear that it's just an ordinary command, ans also signals to someone who isn't sure what it's doing that they can find out just by running `man test`, `help test`, `info test`, etc., from the same shell. I also agree that if statements and if expressions should be kept few and simple. But in some ways it's actually easier to do this in shell languages than in many others! Chaining && and/or || can often get you through a substantial script without any if statements at all, let alone nested ones.
  - xavdid 1y ago
    I mean, there are 3 equally valid ways to write an if statement: `test`, `[`, and `[[`. In the case of the latter two, there are a mess of single-letter flags to test things about a file or condition[0]. I'm not sure what makes them "fake syntax", but I also don't know that much about bash. It's all reasonable enough if you go and look it up, but the script immediately becomes harder to reason about. Conditionals shouldn't be this hard. [0]: https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.ht...
johnisgood 1y ago
I have a couple of projects consisting of around >1k lines of Bash. :) Not to bloat, but it is pretty easy to read and maintain. It is complete as well. I tested all of its functionalities and it just works(tm). Were it another language, it may have been more than just around 1k LOC, however, or more difficult to maintain. I call some external programs a lot, so I stick'd to a shell script.
maccard 1y ago
Unfortunately there’s basically no guarantee that even the simplest scripts work. #!/bin/bash make $1 Has multiple possible problems with it.

pxc 1y ago

When you solve the dependency management issue for shell scripts, you can also use newer language features because you can ship a newer interpreter the same way you ship whatever external dependencies you have. You don't have to limit yourself to what is POSIX, etc. Depending on how you solve it, you may even be able to switch to a newer shell with a nicer language. (And doing so may solve it for you; since PowerShell, newer shells often come with a dependency management layer.) > any script that needs those things It's not really a matter of needing those things, necessarily. Once you have them, you're welcome to write scripts in a cleaner, more convenient way. For instance, all of my shell scripts used by colleagues at work just use GNU coreutils regardless of what platform they're on. Instead of worrying about differences in how sed behaves with certain flags, on different platforms, I simply write everything for GNU sed and it Just Works™. Do those scripts need such a thing? Not necessarily. Is it nicer to write free of constraints like that? Yes! Same thing for just choosing commands with nicer interfaces, or more unified syntax... Use p7zip for handling all your archives so there's only one interface to think about. Make heavy use of `jq` (a great language) for dealing with structured data. Don't worry about reading input from a file and then writing back to it in the same pipeline; just throw in `sponge` from moreutils. > The language just plain sucks too much There really isn't anything better for invoking external programs. Everything else is way clunkier. Maybe that's okay, but when I've rewritten large-ish shell scripts in other languages, I often found myself annoyed with the new language. What used to be a 20-line shell script can easily end up being 400 lines in a "real" language. I kind of agree with you, of course. POSIX-ish shells have too much syntax and at the same time not enough power. But what I really want is a better shell language, not to use some interpreted non-shell language in their place.

MatmaRex 1y ago
Broke: Dependency management used for shell scripts Woke: Dependency management used for installing an interpreter for a better programming language to write your script in it Bespoke: Dependency management used for installing your script
m2f2 1y ago
Nice, if only you could count on having it installed on your fleet, and your fleet is 100pct Linux, no AIX, no HPUX, no SOLARIS, no SUSE on IBM Power.... Been there, tried to, got a huge slap in the face.
- kstrauser 1y ago
  Been there, done that. I am so glad I don’t have to deal with all that insanity anymore. In the build farm I was responsible for, I was always happy to work on the Linux and BSD boxes. AIX and HPUX made me want to throw things. At least the Itanium junk acted like a normal server, just a painfully slow one. I will never voluntarily run a bunch of non-Linux/BSD servers again.
  - lenerdenator 1y ago
    I honestly don't get why there are still a bunch of non-Linux/BSD servers, at least if the goal is to do UNIX-y stuff. I haven't touched AIX or HPUX in probably a decade and I thought they were a weird idea back then: proprietary UNIX? Is it still 1993?
    - kstrauser 1y ago
      At the time (10 years ago) I worked for a company with enormous customers who had all kinds of different deployment targets. I bet that list is a lot shorter today. I hope so, for their sake. shudder

xavdid 1y ago

My rule of thumb is that as soon as I write a conditional, it's time to upgrade bash to Python/Node/etc. I shouldn't have to search for the nuances of `if` statements every time I need to write them.

w0m 1y ago
Historically; my rule of thumb is as soon as I can't see the ~entire script without scrolling - time to rewrite in Python/ansible. I Think about the rewrite, but it usually takes awhile to do it (if ever)
pxc 1y ago
What nuances are there to if statements, exactly? An if statement in, for instance bash, just runs any command and then runs one of two blocks of code based on the exit status of that command. If the exit status is truthy, it runs what follows the `then`. If it's falsey, it rhns what follows the `else`. (`elsif` is admittedly gratuitous syntax— it would be better if it were just implemented as an if inside an else statement.) This seems quite similar to other programming languages and like not very much to remember. I'll admit that one thing I do in my shell scripts is avoid "fake syntax"— I never use `[` or `[[` because these obscure the real structure of the statements for the sake of cuteness. I just write `test`, which makes clear that it's just an ordinary command, ans also signals to someone who isn't sure what it's doing that they can find out just by running `man test`, `help test`, `info test`, etc., from the same shell. I also agree that if statements and if expressions should be kept few and simple. But in some ways it's actually easier to do this in shell languages than in many others! Chaining && and/or || can often get you through a substantial script without any if statements at all, let alone nested ones.
- xavdid 1y ago
  I mean, there are 3 equally valid ways to write an if statement: `test`, `[`, and `[[`. In the case of the latter two, there are a mess of single-letter flags to test things about a file or condition[0]. I'm not sure what makes them "fake syntax", but I also don't know that much about bash. It's all reasonable enough if you go and look it up, but the script immediately becomes harder to reason about. Conditionals shouldn't be this hard. [0]: https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.ht...

johnisgood 1y ago

I have a couple of projects consisting of around >1k lines of Bash. :) Not to bloat, but it is pretty easy to read and maintain. It is complete as well. I tested all of its functionalities and it just works(tm). Were it another language, it may have been more than just around 1k LOC, however, or more difficult to maintain. I call some external programs a lot, so I stick'd to a shell script.

maccard 1y ago

Unfortunately there’s basically no guarantee that even the simplest scripts work. #!/bin/bash make $1 Has multiple possible problems with it.