9 ms·

Introduction to Bash Scripting

151 points by RafelMri 2y ago

jofer 2y ago
For anyone wondering whether or not you really need to know bash: Yes. Yes, you do. Somewhere in the middle of every stack is a bunch of bash and makefiles. Don't replace that or avoid it. Embrace it. It's not perfect, but it's important to know.
- almostgotcaught 2y ago
  I love when people proclaim this as if it's some kind of sage wisdom when it's actually just hypermyopia. For anyone wondering whether this is true: of course it's not since there are plenty of stacks that don't run in environments that are anywhere near a shell.
  - jofer 2y ago
    Can you seriously name something other than very low level embedded software that doesn't involve shell at some level? I certainly can't...
    - almostgotcaught 2y ago
      Do you think your phone runs a shell for IPC? Like passes strings to and fro? How about the runtimes/hypervisors/etc that manage vcpus on AWS? And about that embedded dev - do you think that that's a small niche industry?
      - yjftsjthsd-h 2y ago
        I'm pretty sure Android has a bunch of shell under the hood, and I don't have a jailbroken iOS device to check but I'd be a little surprised if it was different. AWS may or may not, I don't know enough to comment. And yes, about embedded dev - there's the very very low level stuff of course, but the moment you get above tiny microcontrollers it's all Linux again and there's shell everywhere. Actually, even for the really tiny embedded stuff I'd still expect to find a mess of shell scripts, just in the build environment rather than on-device.
        almostgotcaught 2y ago
        > I'm pretty sure Android has a bunch of shell under the hood I didn't claim there was no shell - I claimed that IPC isn't performed with pipes.
        yjftsjthsd-h 2y ago
        And the kernel also doesn't run shell, but the claim was that you would struggle to find > software that doesn't involve shell at some level which would seem to me to encompass more of the stack. Like... okay, Android IPC isn't pipe-based. Does that release anyone from touching shell? Anyone working on a ROM is going to get their hands dirty with the rest of the OS. And I struggle to believe that any app developer is avoiding it (it's always the build step that does you in). Approximately nobody is working on just IPC in isolation.
        almostgotcaught 2y ago
        > it's always the build step that does you in Just curious: where exactly is the shell in `cmake -GNinja`? Or is CMake not a build system hmmm? Nevermind that some people use bazel or meson or something ... other than shell... Shell scripts in your build system are a code smell. In cpp projects it's covering up for people that don't how to use the actually correct tool for the job: the CMake scripting language.
        yjftsjthsd-h 2y ago
        > Just curious: where exactly is the shell in `cmake -GNinja`? Or is CMake not a build system hmmm? Nevermind that some people use bazel or meson or something ... other than shell... Right above it; cmake replaces make, but something still runs it. And IME that something is usually either a developer running it from a shell (ideally eventually with another shell command wrapping it to automatically build on file changes) or a CI system running it which is always a massive pile of shell scripts. To be fair, I don't doubt that if you really tried you could build a dev environment that used zero shell. But you would have to really try, and half of the things you had to build to make it work would inevitably converge to an informally-specified, bug-ridden, slow implementation of half of Common Lisp^w^wBASH.
      - jofer 2y ago
        Shell pops up in a ton of places even where it's not used directly. Build systems are a good example. I have yet to come across something that doesn't touch shell at some level unless it also doesn't touch an OS. But maybe that's just me. Phones aren't an exception to that, though.
      - wiseowise 2y ago
        Those things might not run bash/sh, but I’m willing to bet a hundred quid that nearly all of their development environments had at least one tiny shell script that did some transformation of data.
  - wpm 2y ago
    Whether they run in environments where a shell is nearby has no bearing on whether they are built or tested in environments where a shell is near and sufficient for the task.
    - almostgotcaught 2y ago
      Pasting what I responded to the other person talking about build systems: Just curious: where exactly is the shell in `cmake -GNinja`? Or is CMake not a build system hmmm? Nevermind that some people use bazel or meson or something ... other than shell... Shell scripts in your build system are a code smell. In cpp projects it's covering up for people that don't how to use the actually correct tool for the job: the CMake scripting language.
- benreesman 2y ago
  Some things change: you once needed Perl where you now need Python. Other things don’t: you need non-trivial bash. I don’t mind if people do their heavy lifting in VSCode, I like the tool myself. But if you can’t unfuck the little daemon that VSCode talks to? You’re out of commission. You’re at the mercy of people who can.
- deleted 2y ago
  [deleted]
udev4096 2y ago
This looks extremely basic. I would instead recommend reading https://bash.academy https://bash.academy or TLDP Advance Bash Scripting Guide
- deleted 2y ago
  [deleted]
- imp0cat 2y ago
  That's a good thing, isn't it? It's not too scary for someone who's not well versed in shell scripting. And you can later move on to other resources (TLDP is great!).
  - wyclif 2y ago
    I looked at it this morning and I think it's helpful for those brand new to the shell. My only criticism of it is that it needs a once-over edit by a native English speaker. The exercises seem fine to me.
callalex 2y ago
Can someone name me examples of where a bash script is actually the _correct_ tool for the job in modern times? Or is it still used only because it’s the easily available tool?
- jofer 2y ago
  Any time you're running lots of CLI utilities directly is a good start. Bash is very very good for that, and other than perl, most other things aren't particularly good at it.
  - lambdaba 2y ago
    Ruby is the greatest at it and inline deps make it perfect for scripting.
  - jcranmer 2y ago
    Unless you need to run programs with arguments with spaces in their name. Not that long ago, I needed to write a script that would grab the last command in a shell file that had a list of commands, add an argument or two to it, and run it. Which proceeded to barf because the command had quotes, for some of the arguments had necessary spaces in them, and I couldn't figure out the way to get the necessary string in a way that bash could properly execute the command. My eventual solution was to just use python's shlex module to parse the line and output an executable command (and remark that I should have just started the script in Python in the first place)...
    - PhilipRoman 2y ago
      >Unless you need to run programs with arguments with spaces in their name. It does that just fine, I don't have all the details obviously but it sounds like printf %q or ${arg@Q} is your friend here.
      - wiseowise 2y ago
        > ${arg@Q} Right, how couldn’t they know to type actual gibberish to make it work.
throwaway290232 2y ago
[flagged]
rty32 2y ago
I know basic bash and can read most bash scripts, but never spent much time on it to become an expert. However I found ChatGPT to be extremely valuable -- given the description of a task, it can create very high quality scripts with best practice and comments, often correctly using third party tools. That's probably what I'll do going forward -- my time is better spent creating a good prompt and reviewing output.
- dools 2y ago
  Same, and another great thing I have been using since ChatGPT made it trivial to do so is systemd. I was using pm2 quite a bit for running scripts but high quality bash (or python or php or whatever really) plus systemd is really killer. I’ve also been doing way more with google cloud shell. I was always such a point and click guy when it came to setting stuff up in GCP but automating and scripting with bash in cloud shell using ChatGPT is so much faster and repeatable.
- wpm 2y ago
  Depends on the platform. macOS has a terminal and ChatGPT is highly Linux biased, and it's quite happy to spew trash when you point out "hey bozo I said macOS useradd isn't a real command on macOS".
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- TechDebtDevin 2y ago
  I'm pretty sure most llms have very low eval scores on bash compared to other common languages.
  - DonHopkins 2y ago
    And most humans have MUCH MUCH lower eval scores on bash, yet they write it anyway, and the rest of us get stuck with trying to use and understand and debug and maintain that crap, that bursts into fire and explosive diarrhea whenever it hits and edge condition or file name with a space in it.
asicsp 2y ago
See also https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide and https://github.com/denysdovhan/bash-handbook https://github.com/denysdovhan/bash-handbook. https://www.shellcheck.net/ https://www.shellcheck.net/ is amazing to spot pitfalls. See my list https://learnbyexample.github.io/curated_resources/linux_cli_scripting.html https://learnbyexample.github.io/curated_resources/linux_cli... for learning resources for CLI tools, scripting, etc.
jimmar 2y ago
I've coded in a bunch of languages over my career, but somehow managed not to use bash. But a few weeks ago I found myself needing to create a somewhat involved script using Bash. Using GitHub Copilot, I was able to generate a script that worked great and it included comments, arrays, arguments, case statements, loops, conditionals, menus, etc--basically covering the first dozen chapters of this book. I guess my point is that we need to teach people what's possible and to help them explain what they want clearly. Memorizing syntax will be a less relevant skill at some point.
- guappa 2y ago
  I haven't seen your script but I bet it will break in a hundreds corner cases.
qz_kb 2y ago
Using python and constraining yourself to only use a basic subset of the standard library modules so you can run the script in pretty much any environment is almost always a better choice than trying to write even one loop, if statement, or argument parser in a bash script. bash script is "okay" I guess if your "script" is just a series of commands with no control flow.
- isbvhodnvemrwvn 2y ago
  I would agree if python dependency management wasn't a dumpster fire.
  - rendaw 2y ago
    I think the point GP was making was that you restrict yourself to only the bundled standard library, which covers most of the basics needed for scripting.
  - qz_kb 2y ago
    This is why you force yourself to use nearly zero dependencies. The standard library sys, os, subprocess, and argparse modules should be all you need to do all the fancy stuff you might try with bash, and have extremely high compatibility with any python3.x install.
  - guappa 2y ago
    And it's a dumpster fire because they refuse to make any decision and decide which is the supported way. Instead they removed distutils, so now there is no way to install any module without using a 3rd party installer.
- rendaw 2y ago
  I agree, but I've been having a hard time even with python recently. I had a small script (50-100 lines) to format a data drive on boot I refactored, 3 or 4 obvious undeclared variables and who knows how many more I didn't notice - mypy found 0 issues. I was looking up statically typed alternatives and stumbled upon Ammonite and Scala-CLI for scala. I haven't used them much, but Ammonite bundles some basic libraries including command line parsing, http, and json, which are probably 99% of what I used in Python too? And Scala seems like an interesting language too with decent editor integration.
baxuz 2y ago
Tried https://github.com/google/zx https://github.com/google/zx and never looked back.
- pindab0ter 2y ago
  That's not available on basically every system. What advantages does zx have over Python, Ruby or even Fish for example, if we're installing extra programming languages?
  - baxuz 2y ago
    It's what fits my use case quite well.
- vwkd 2y ago
  How about dax[^1] which is cross-platform and also works with Deno? [^1]: https://github.com/dsherret/dax https://github.com/dsherret/dax
ayakang31415 2y ago
Is there anything Bash does that Python can't do? I feel like using Python is just way better than using Bash.
- wpm 2y ago
  Work out of the box on any macOS/Linux computer.
  - ayakang31415 2y ago
    You use Bash for that not because it is better, but because UNIX related binaries are configured in Bash.
- cassianoleal 2y ago
  Is there anything Python does that Bash can't do? There are many reasons to use either. Probably about as many as there are to use the other. Choose wisely.
  - ayakang31415 2y ago
    Yeah, lots of things Python can do but Bash cannot simply because of vast selection of third party modules available in Python.
    - cassianoleal 2y ago
      I agree that something which requires a "vast selection of third party modules" is probably not the kind of thing you'd do in Bash. That's a strawman though. I don't want to have to install a "vast selection of third party modules" just so I can run `kubectl apply -f foobar/*` on my CI pipeline.
macrael 2y ago
Write scripts in whatever tooling you have setup for your app. In a typescript codebase, I write typescript scripts. In a go codebase I write go scripts. It's so easy to write a couple wrappers to make execing something a one liner and then the advantages of bash have disappeared. You already have a toolchain for running a program in your language, just use that to build your scripts.
xlii 2y ago
Recently I’ve been exploring back Perl. I loved it many years ago and today I just need “glue”. It seems to be quite fast, can escape UNIX-Linux idiosyncrasies and I still can see all the magic variables in the code after those years. Maybe it’s better to just get back into Perl instead of figuring out how to pipe back to back to few applications and jq on top?
- kamaal 2y ago
  If you are using Bash, you probably need to use Perl. Python unfortunately doesn't shine where Perl is good at. Mostly of the command line work involves passing around adhoc text, parsing and using it. You are better off using a language designed to do that very task than use something else.
tjoff 2y ago
A lot of people suggests python, I'd argue that python is a bad fit for what many bash scripts does. A bash script is basically scripting what you do on your terminal. I feel like scripting in the same language that is your terminal interface is underrated. You can basically take a look at your command history and wrap it up as a script. Yes, bash is clunky as hell. It has some really ugly warts. But learning the basics will make you more proficient in the terminal. The synergies are really nice.
- _xiaz 2y ago
  As a linux admin, bash and python are my bread and butter. I have two heuristics for choosing between them: - If it needs more math than counting, choose Python - If it's >200 lines (500 if I'm brave), choose Python Otherwise, shell scripting is nice
  - rcxdude 2y ago
    my heuristic is "if there's an if statement, loop, or variable substitution with uncontrolled input, use python".
- lloeki 2y ago
  Interesting that a lot of the (general, not specifically here) commentary is "don't use bash, use python" which I find quite clunky, noisy, and boilerplatey to do the same thing as a shell script. Kinda Master Foo and the Ten Thousand Lines†. Ruby's https://github.com/ruby/shell https://github.com/ruby/shell (which used to be bundled in Ruby itself) doesn't get much mention but is much more clear and useful to me. # system commands are not defined by default, only builtins, for safety Shell.def_system_command 'ls' Shell.def_system_command 'grep' Shell.def_system_command 'jq' Shell.new.transact do # simple redirs just work ls | grep('test') > 'some/file' ls | grep('other') | tee('foo') >> 'some/file' # pops at the end of the block cd 'some/dir' do # nested redir for demo (cat < 'foo.json') | jq '.[] | { bar: .bar }' end end † http://www.catb.org/esr/writings/unix-koans/ten-thousand.html http://www.catb.org/esr/writings/unix-koans/ten-thousand.htm...
TheLoafOfBread 2y ago
Or you can use PowerShell - Available on MacOS, Windows, Linux instead of suffering through incoherent syntax of Bash.
udev4096 2y ago
If anyone's looking to write tests for bash scripts, I wrote a small blog post few weeks ago: https://hcrypt.net/test.html https://hcrypt.net/test.html
alwinaugustin 2y ago
I am new to bash and always struck at using the tools like `awk` and `sed.` You need to be good at regular expressions to write a good script using these tools.
pronoiac 2y ago
I recently was running a variety of compressors over the course of days, and using Bash was great because, it doesn't parse the whole file before it starts running - I could edit and reprioritize next steps while the script was running. "Oh, that one's not working, skip the slower version" or "oh, space is an issue, work on freeing that up." It's a weird hack I wouldn't suggest in production, but it worked well for me.
foreigner 2y ago
Debating the pros and cons of different languages misses the point IMO. Shell scripts are useful not because of the language but because the interpreter is already available absolutely everywhere.
1vuio0pswjnm7 2y ago
Bash is not a good shell for scripting. Too big and slow.
bobbyiliev 2y ago
Many people are saying that ChatGPT or Copilot could write great scripts which is indeed true, but the eBook was written way before those LLMs were a thing. It is still good to learn the basics though.
ngcc_hk 2y ago
Bash scripting is hard ... missing a space or add a space ... How do you debug it and trace it ... I do use it as said this is the terminal command line and both Mac/Linux supported it. And if it works, it work even nohup. Just hard.
- kchr 2y ago
  You debug it with shellcheck[0] and `set -ex` (which will trace each statement on STDOUT). [0] https://www.shellcheck.net/ https://www.shellcheck.net/
Too 2y ago
Just skimmed through the book and noticed several examples where it doesn't use argument quoting. For example the first chapter about variables doing echo $name. In fact, there is no mention of argument quoting at all. This should be the number one lesson before touching any bash script. Otherwise your scripts will be vulnerable to command injection. I recommend running the whole book through shellcheck. It also recommends wrapping variables with curly braces like ${name}, without explaining why, later on never does so itself throughout the rest of the book. There are a lot of such missing explanations in the whole book. It does make it short and fun to jump straight into the examples but i think it could benefit from some facts and reference material, at least to link to. Anyway, seems like most people here are more interested in arguing over bash vs python than discussing the actual content. A little surprising to see this get so many points.

151 points by RafelMri 2y ago

For anyone wondering whether or not you really need to know bash: Yes. Yes, you do. Somewhere in the middle of every stack is a bunch of bash and makefiles. Don't replace that or avoid it. Embrace it. It's not perfect, but it's important to know.

almostgotcaught 2y ago
I love when people proclaim this as if it's some kind of sage wisdom when it's actually just hypermyopia. For anyone wondering whether this is true: of course it's not since there are plenty of stacks that don't run in environments that are anywhere near a shell.
- jofer 2y ago
  Can you seriously name something other than very low level embedded software that doesn't involve shell at some level? I certainly can't...
  - almostgotcaught 2y ago
    Do you think your phone runs a shell for IPC? Like passes strings to and fro? How about the runtimes/hypervisors/etc that manage vcpus on AWS? And about that embedded dev - do you think that that's a small niche industry?
    - yjftsjthsd-h 2y ago
      I'm pretty sure Android has a bunch of shell under the hood, and I don't have a jailbroken iOS device to check but I'd be a little surprised if it was different. AWS may or may not, I don't know enough to comment. And yes, about embedded dev - there's the very very low level stuff of course, but the moment you get above tiny microcontrollers it's all Linux again and there's shell everywhere. Actually, even for the really tiny embedded stuff I'd still expect to find a mess of shell scripts, just in the build environment rather than on-device.
      - almostgotcaught 2y ago
        > I'm pretty sure Android has a bunch of shell under the hood I didn't claim there was no shell - I claimed that IPC isn't performed with pipes.
        yjftsjthsd-h 2y ago
        And the kernel also doesn't run shell, but the claim was that you would struggle to find > software that doesn't involve shell at some level which would seem to me to encompass more of the stack. Like... okay, Android IPC isn't pipe-based. Does that release anyone from touching shell? Anyone working on a ROM is going to get their hands dirty with the rest of the OS. And I struggle to believe that any app developer is avoiding it (it's always the build step that does you in). Approximately nobody is working on just IPC in isolation.
        almostgotcaught 2y ago
        > it's always the build step that does you in Just curious: where exactly is the shell in `cmake -GNinja`? Or is CMake not a build system hmmm? Nevermind that some people use bazel or meson or something ... other than shell... Shell scripts in your build system are a code smell. In cpp projects it's covering up for people that don't how to use the actually correct tool for the job: the CMake scripting language.
        yjftsjthsd-h 2y ago
        > Just curious: where exactly is the shell in `cmake -GNinja`? Or is CMake not a build system hmmm? Nevermind that some people use bazel or meson or something ... other than shell... Right above it; cmake replaces make, but something still runs it. And IME that something is usually either a developer running it from a shell (ideally eventually with another shell command wrapping it to automatically build on file changes) or a CI system running it which is always a massive pile of shell scripts. To be fair, I don't doubt that if you really tried you could build a dev environment that used zero shell. But you would have to really try, and half of the things you had to build to make it work would inevitably converge to an informally-specified, bug-ridden, slow implementation of half of Common Lisp^w^wBASH.
    - jofer 2y ago
      Shell pops up in a ton of places even where it's not used directly. Build systems are a good example. I have yet to come across something that doesn't touch shell at some level unless it also doesn't touch an OS. But maybe that's just me. Phones aren't an exception to that, though.
    - wiseowise 2y ago
      Those things might not run bash/sh, but I’m willing to bet a hundred quid that nearly all of their development environments had at least one tiny shell script that did some transformation of data.
- wpm 2y ago
  Whether they run in environments where a shell is nearby has no bearing on whether they are built or tested in environments where a shell is near and sufficient for the task.
  - almostgotcaught 2y ago
    Pasting what I responded to the other person talking about build systems: Just curious: where exactly is the shell in `cmake -GNinja`? Or is CMake not a build system hmmm? Nevermind that some people use bazel or meson or something ... other than shell... Shell scripts in your build system are a code smell. In cpp projects it's covering up for people that don't how to use the actually correct tool for the job: the CMake scripting language.
benreesman 2y ago
Some things change: you once needed Perl where you now need Python. Other things don’t: you need non-trivial bash. I don’t mind if people do their heavy lifting in VSCode, I like the tool myself. But if you can’t unfuck the little daemon that VSCode talks to? You’re out of commission. You’re at the mercy of people who can.
deleted 2y ago
[deleted]

udev4096 2y ago

This looks extremely basic. I would instead recommend reading https://bash.academy https://bash.academy or TLDP Advance Bash Scripting Guide

deleted 2y ago
[deleted]
imp0cat 2y ago
That's a good thing, isn't it? It's not too scary for someone who's not well versed in shell scripting. And you can later move on to other resources (TLDP is great!).
- wyclif 2y ago
  I looked at it this morning and I think it's helpful for those brand new to the shell. My only criticism of it is that it needs a once-over edit by a native English speaker. The exercises seem fine to me.

callalex 2y ago

Can someone name me examples of where a bash script is actually the _correct_ tool for the job in modern times? Or is it still used only because it’s the easily available tool?

jofer 2y ago
Any time you're running lots of CLI utilities directly is a good start. Bash is very very good for that, and other than perl, most other things aren't particularly good at it.
- lambdaba 2y ago
  Ruby is the greatest at it and inline deps make it perfect for scripting.
- jcranmer 2y ago
  Unless you need to run programs with arguments with spaces in their name. Not that long ago, I needed to write a script that would grab the last command in a shell file that had a list of commands, add an argument or two to it, and run it. Which proceeded to barf because the command had quotes, for some of the arguments had necessary spaces in them, and I couldn't figure out the way to get the necessary string in a way that bash could properly execute the command. My eventual solution was to just use python's shlex module to parse the line and output an executable command (and remark that I should have just started the script in Python in the first place)...
  - PhilipRoman 2y ago
    >Unless you need to run programs with arguments with spaces in their name. It does that just fine, I don't have all the details obviously but it sounds like printf %q or ${arg@Q} is your friend here.
    - wiseowise 2y ago
      > ${arg@Q} Right, how couldn’t they know to type actual gibberish to make it work.

throwaway290232 2y ago

[flagged]

rty32 2y ago

I know basic bash and can read most bash scripts, but never spent much time on it to become an expert. However I found ChatGPT to be extremely valuable -- given the description of a task, it can create very high quality scripts with best practice and comments, often correctly using third party tools. That's probably what I'll do going forward -- my time is better spent creating a good prompt and reviewing output.

dools 2y ago
Same, and another great thing I have been using since ChatGPT made it trivial to do so is systemd. I was using pm2 quite a bit for running scripts but high quality bash (or python or php or whatever really) plus systemd is really killer. I’ve also been doing way more with google cloud shell. I was always such a point and click guy when it came to setting stuff up in GCP but automating and scripting with bash in cloud shell using ChatGPT is so much faster and repeatable.
wpm 2y ago
Depends on the platform. macOS has a terminal and ChatGPT is highly Linux biased, and it's quite happy to spew trash when you point out "hey bozo I said macOS useradd isn't a real command on macOS".
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
TechDebtDevin 2y ago
I'm pretty sure most llms have very low eval scores on bash compared to other common languages.
- DonHopkins 2y ago
  And most humans have MUCH MUCH lower eval scores on bash, yet they write it anyway, and the rest of us get stuck with trying to use and understand and debug and maintain that crap, that bursts into fire and explosive diarrhea whenever it hits and edge condition or file name with a space in it.

asicsp 2y ago

See also https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide and https://github.com/denysdovhan/bash-handbook https://github.com/denysdovhan/bash-handbook. https://www.shellcheck.net/ https://www.shellcheck.net/ is amazing to spot pitfalls. See my list https://learnbyexample.github.io/curated_resources/linux_cli_scripting.html https://learnbyexample.github.io/curated_resources/linux_cli... for learning resources for CLI tools, scripting, etc.

jimmar 2y ago

I've coded in a bunch of languages over my career, but somehow managed not to use bash. But a few weeks ago I found myself needing to create a somewhat involved script using Bash. Using GitHub Copilot, I was able to generate a script that worked great and it included comments, arrays, arguments, case statements, loops, conditionals, menus, etc--basically covering the first dozen chapters of this book. I guess my point is that we need to teach people what's possible and to help them explain what they want clearly. Memorizing syntax will be a less relevant skill at some point.

guappa 2y ago
I haven't seen your script but I bet it will break in a hundreds corner cases.

qz_kb 2y ago

Using python and constraining yourself to only use a basic subset of the standard library modules so you can run the script in pretty much any environment is almost always a better choice than trying to write even one loop, if statement, or argument parser in a bash script. bash script is "okay" I guess if your "script" is just a series of commands with no control flow.

isbvhodnvemrwvn 2y ago
I would agree if python dependency management wasn't a dumpster fire.
- rendaw 2y ago
  I think the point GP was making was that you restrict yourself to only the bundled standard library, which covers most of the basics needed for scripting.
- qz_kb 2y ago
  This is why you force yourself to use nearly zero dependencies. The standard library sys, os, subprocess, and argparse modules should be all you need to do all the fancy stuff you might try with bash, and have extremely high compatibility with any python3.x install.
- guappa 2y ago
  And it's a dumpster fire because they refuse to make any decision and decide which is the supported way. Instead they removed distutils, so now there is no way to install any module without using a 3rd party installer.
rendaw 2y ago
I agree, but I've been having a hard time even with python recently. I had a small script (50-100 lines) to format a data drive on boot I refactored, 3 or 4 obvious undeclared variables and who knows how many more I didn't notice - mypy found 0 issues. I was looking up statically typed alternatives and stumbled upon Ammonite and Scala-CLI for scala. I haven't used them much, but Ammonite bundles some basic libraries including command line parsing, http, and json, which are probably 99% of what I used in Python too? And Scala seems like an interesting language too with decent editor integration.

baxuz 2y ago

Tried https://github.com/google/zx https://github.com/google/zx and never looked back.

pindab0ter 2y ago
That's not available on basically every system. What advantages does zx have over Python, Ruby or even Fish for example, if we're installing extra programming languages?
- baxuz 2y ago
  It's what fits my use case quite well.
vwkd 2y ago
How about dax[^1] which is cross-platform and also works with Deno? [^1]: https://github.com/dsherret/dax https://github.com/dsherret/dax

ayakang31415 2y ago

Is there anything Bash does that Python can't do? I feel like using Python is just way better than using Bash.

wpm 2y ago
Work out of the box on any macOS/Linux computer.
- ayakang31415 2y ago
  You use Bash for that not because it is better, but because UNIX related binaries are configured in Bash.
cassianoleal 2y ago
Is there anything Python does that Bash can't do? There are many reasons to use either. Probably about as many as there are to use the other. Choose wisely.
- ayakang31415 2y ago
  Yeah, lots of things Python can do but Bash cannot simply because of vast selection of third party modules available in Python.
  - cassianoleal 2y ago
    I agree that something which requires a "vast selection of third party modules" is probably not the kind of thing you'd do in Bash. That's a strawman though. I don't want to have to install a "vast selection of third party modules" just so I can run `kubectl apply -f foobar/*` on my CI pipeline.

macrael 2y ago

Write scripts in whatever tooling you have setup for your app. In a typescript codebase, I write typescript scripts. In a go codebase I write go scripts. It's so easy to write a couple wrappers to make execing something a one liner and then the advantages of bash have disappeared. You already have a toolchain for running a program in your language, just use that to build your scripts.

xlii 2y ago

Recently I’ve been exploring back Perl. I loved it many years ago and today I just need “glue”. It seems to be quite fast, can escape UNIX-Linux idiosyncrasies and I still can see all the magic variables in the code after those years. Maybe it’s better to just get back into Perl instead of figuring out how to pipe back to back to few applications and jq on top?

kamaal 2y ago
If you are using Bash, you probably need to use Perl. Python unfortunately doesn't shine where Perl is good at. Mostly of the command line work involves passing around adhoc text, parsing and using it. You are better off using a language designed to do that very task than use something else.

tjoff 2y ago

A lot of people suggests python, I'd argue that python is a bad fit for what many bash scripts does. A bash script is basically scripting what you do on your terminal. I feel like scripting in the same language that is your terminal interface is underrated. You can basically take a look at your command history and wrap it up as a script. Yes, bash is clunky as hell. It has some really ugly warts. But learning the basics will make you more proficient in the terminal. The synergies are really nice.

_xiaz 2y ago
As a linux admin, bash and python are my bread and butter. I have two heuristics for choosing between them: - If it needs more math than counting, choose Python - If it's >200 lines (500 if I'm brave), choose Python Otherwise, shell scripting is nice
- rcxdude 2y ago
  my heuristic is "if there's an if statement, loop, or variable substitution with uncontrolled input, use python".
lloeki 2y ago
Interesting that a lot of the (general, not specifically here) commentary is "don't use bash, use python" which I find quite clunky, noisy, and boilerplatey to do the same thing as a shell script. Kinda Master Foo and the Ten Thousand Lines†. Ruby's https://github.com/ruby/shell https://github.com/ruby/shell (which used to be bundled in Ruby itself) doesn't get much mention but is much more clear and useful to me. # system commands are not defined by default, only builtins, for safety Shell.def_system_command 'ls' Shell.def_system_command 'grep' Shell.def_system_command 'jq' Shell.new.transact do # simple redirs just work ls | grep('test') > 'some/file' ls | grep('other') | tee('foo') >> 'some/file' # pops at the end of the block cd 'some/dir' do # nested redir for demo (cat < 'foo.json') | jq '.[] | { bar: .bar }' end end † http://www.catb.org/esr/writings/unix-koans/ten-thousand.html http://www.catb.org/esr/writings/unix-koans/ten-thousand.htm...

TheLoafOfBread 2y ago

Or you can use PowerShell - Available on MacOS, Windows, Linux instead of suffering through incoherent syntax of Bash.

udev4096 2y ago

If anyone's looking to write tests for bash scripts, I wrote a small blog post few weeks ago: https://hcrypt.net/test.html https://hcrypt.net/test.html

alwinaugustin 2y ago

I am new to bash and always struck at using the tools like `awk` and `sed.` You need to be good at regular expressions to write a good script using these tools.

pronoiac 2y ago

I recently was running a variety of compressors over the course of days, and using Bash was great because, it doesn't parse the whole file before it starts running - I could edit and reprioritize next steps while the script was running. "Oh, that one's not working, skip the slower version" or "oh, space is an issue, work on freeing that up." It's a weird hack I wouldn't suggest in production, but it worked well for me.

foreigner 2y ago

Debating the pros and cons of different languages misses the point IMO. Shell scripts are useful not because of the language but because the interpreter is already available absolutely everywhere.

1vuio0pswjnm7 2y ago

Bash is not a good shell for scripting. Too big and slow.

bobbyiliev 2y ago

Many people are saying that ChatGPT or Copilot could write great scripts which is indeed true, but the eBook was written way before those LLMs were a thing. It is still good to learn the basics though.

ngcc_hk 2y ago

Bash scripting is hard ... missing a space or add a space ... How do you debug it and trace it ... I do use it as said this is the terminal command line and both Mac/Linux supported it. And if it works, it work even nohup. Just hard.

kchr 2y ago
You debug it with shellcheck[0] and `set -ex` (which will trace each statement on STDOUT). [0] https://www.shellcheck.net/ https://www.shellcheck.net/

Too 2y ago

Just skimmed through the book and noticed several examples where it doesn't use argument quoting. For example the first chapter about variables doing echo $name. In fact, there is no mention of argument quoting at all. This should be the number one lesson before touching any bash script. Otherwise your scripts will be vulnerable to command injection. I recommend running the whole book through shellcheck. It also recommends wrapping variables with curly braces like ${name}, without explaining why, later on never does so itself throughout the rest of the book. There are a lot of such missing explanations in the whole book. It does make it short and fun to jump straight into the examples but i think it could benefit from some facts and reference material, at least to link to. Anyway, seems like most people here are more interested in arguing over bash vs python than discussing the actual content. A little surprising to see this get so many points.