4 ms·

Once I type a square bracket in a shell script, I know it's time to bring out Python. I don't think it's humanely possible to write a test command without shoot

Once I type a square bracket in a shell script, I know it's time to bring out Python. I don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way.

usr1106 6y ago
> don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way That's a weird claim. Could it be that you have never really learned shell programming or read the man pages? Or can you give an example? We have lot of production code with many test statements and I don't remember when we were bitten last time by some "surprising feature" of test. I agree that test works differently than comparisions in many programming languages. So you need to understand how it is different. And you need to be disciplined about quoting, otherwise surprises will happen when you have user-controlled input and the shell starts to do word-splitting. But if you use shellcheck you will be reminded of quoting everywhere. It's not a problem specific to test anyway.
- imtringued 6y ago
  I have written dozens of of bash utilities at a single company. Probably 5000 lines of bash in total. I still don't know the whole syntax for if/for/case and have to look it up every single time.
  - zimpenfish 6y ago
    I've written hundreds (if not thousands) of bash utilities over 20+ years and I still occasionally have to check the syntax for various bits.
- kristaps 6y ago
  > So you need to understand how it is different. And you need to be disciplined about quoting Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot.
- meowface 6y ago
  If I'm modifying an existing script, I generally have no problem doing that. But the tweet quoted in the article rings very true to me: >The opposite of "it's like riding a bike" is "it's like programming in bash". >A phrase which means that no matter how many times you do something, you will have to re-learn it every single time. I and probably a lot of programmers write bash scripts just enough to remember the vague shape of what we want to do, yet still infrequently enough to require looking up the exact characters every single time (-z? ! -z? one bracket? two brackets? quotes? no quotes? something about a semicolon and "then", right? where can you add newlines, again? oh yeah, it's not "end" like Ruby/Lua but "fi" right? what's "else if", again?). I'm sure if we were working with bash scripts daily, we'd memorize it, but it's just rare and just different enough that it's basically day 1 all over again every time. Shellcheck helps as a safety net, but it doesn't necessarily save me a lot of time and cognitive load. If I'm writing a new script from scratch and I realize there's going to be some branching or prompts or complex argument handling, I just write it in Python. It saves time and effort, it lets other people on my team more quickly modify and debug it, and it's more predictably portable ("oops, only sh, no bash"). And if it ever needs to do something more advanced in the future, it's way easier to extend. Or put much more succinctly, by 'kristaps in another comment: >Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot. It's nice for quickly writing something that's just wrapping multiple commands, though.
  - zdw 6y ago
    One of the big points of linters is that they change developer behavior for the better over time. Once you get used to using shellcheck all the time your code style starts to match what it requires. You start thinking about quoting and other intricacies, and coding much more defensively. This happens with other checkers as well - >90%+ of the time the python I write passes pep8 and black cleanly after having used those tools for a year or two.
    - rovr138 6y ago
      Using Python how frequently? The issue is the frequency. Due to not coding in it often, you don't retain it.
- radiowave 6y ago
  I tend to apply much the same rule as emptyparadise - once conditional logic appears, it's time for a rethink. Part of my rationale for this is that I assume that I'm dealing with the thin end of the wedge - if there's enough complexity in the task to warrant conditionals, there's likely a whole bunch more (or there will be in future) that I haven't encountered or grasped yet. I'm thus willing to take the time-hit on rewriting in Python, because I'm betting that resulting code will be easier to adapt and easier to troubleshoot as this extra complexity gradually comes into view.
  - spurgu 6y ago
    I just can't fathom this as I've never really had compatibility issues/quirks with good old #!/bin/bash. Only issues I've had have been with sed on Mac vs Linux. And I've written thousands of lines with lots of logic, generally breaking statements that become nested into separate functions. Sometimes some things become too complicated for Bash, like parsing JSON, in which case I prefer to break out and call a Python or Node script instead of wrapping my head around jq. But my scripts are also either for personal use, or for internal use (5 person team) at work, so I know who will be running the scripts, why, and where (which OS/versions). Suppose it's a bit like developing for Apple where you can remain aware of the use (and edge) cases.
dexen 6y ago
Awk is a viable replacement for Bash for scripting. A long established standard and installed on pretty much every POSIX system. It has more robust facilities for handling variables, arrays, regex search & substitution, etc.
- usr1106 6y ago
  awk is a full programming language. But it has gotten little attention in the last 20 years. If you expect that someone else can read and maintain your code with minimal learning I don't think it's the tool of choice.
  - jrochkind1 6y ago
    But... neither is bash. I am sympathetic to the argument that awk is theoretically superior to bash in almost all ways (and it is of course nearly as available as bash). But... something still makes me reluctant to switch to using it in the cases I write a bash script (which I'm already trying to minimize, as is the OP). I'm not really sure why though. (Fun-for-mostly-me fact, my very first programming job ever was writing awk. It was a while ago. I certainly don't remember any awk now).
- alexhutcheson 6y ago
  Awk is great for snippets that are meant to be copy+pasted into a shell, and is the only reasonable option sometimes if you're stuck in a Busybox environment, but for almost everything else you'd be better off with Python. This previous comment of mine goes into a bit more detail: https://news.ycombinator.com/item?id=23242753 https://news.ycombinator.com/item?id=23242753
- fomine3 6y ago
  Agreed but I point that not all awks are gawk.
vlowther 6y ago
Or better yet, condition yourself to hit [ a second time and use Bash's built-in conditionals instead of test and [

usr1106 6y ago

> don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way That's a weird claim. Could it be that you have never really learned shell programming or read the man pages? Or can you give an example? We have lot of production code with many test statements and I don't remember when we were bitten last time by some "surprising feature" of test. I agree that test works differently than comparisions in many programming languages. So you need to understand how it is different. And you need to be disciplined about quoting, otherwise surprises will happen when you have user-controlled input and the shell starts to do word-splitting. But if you use shellcheck you will be reminded of quoting everywhere. It's not a problem specific to test anyway.

imtringued 6y ago
I have written dozens of of bash utilities at a single company. Probably 5000 lines of bash in total. I still don't know the whole syntax for if/for/case and have to look it up every single time.
- zimpenfish 6y ago
  I've written hundreds (if not thousands) of bash utilities over 20+ years and I still occasionally have to check the syntax for various bits.
kristaps 6y ago
> So you need to understand how it is different. And you need to be disciplined about quoting Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot.
meowface 6y ago
If I'm modifying an existing script, I generally have no problem doing that. But the tweet quoted in the article rings very true to me: >The opposite of "it's like riding a bike" is "it's like programming in bash". >A phrase which means that no matter how many times you do something, you will have to re-learn it every single time. I and probably a lot of programmers write bash scripts just enough to remember the vague shape of what we want to do, yet still infrequently enough to require looking up the exact characters every single time (-z? ! -z? one bracket? two brackets? quotes? no quotes? something about a semicolon and "then", right? where can you add newlines, again? oh yeah, it's not "end" like Ruby/Lua but "fi" right? what's "else if", again?). I'm sure if we were working with bash scripts daily, we'd memorize it, but it's just rare and just different enough that it's basically day 1 all over again every time. Shellcheck helps as a safety net, but it doesn't necessarily save me a lot of time and cognitive load. If I'm writing a new script from scratch and I realize there's going to be some branching or prompts or complex argument handling, I just write it in Python. It saves time and effort, it lets other people on my team more quickly modify and debug it, and it's more predictably portable ("oops, only sh, no bash"). And if it ever needs to do something more advanced in the future, it's way easier to extend. Or put much more succinctly, by 'kristaps in another comment: >Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot. It's nice for quickly writing something that's just wrapping multiple commands, though.
- zdw 6y ago
  One of the big points of linters is that they change developer behavior for the better over time. Once you get used to using shellcheck all the time your code style starts to match what it requires. You start thinking about quoting and other intricacies, and coding much more defensively. This happens with other checkers as well - >90%+ of the time the python I write passes pep8 and black cleanly after having used those tools for a year or two.
  - rovr138 6y ago
    Using Python how frequently? The issue is the frequency. Due to not coding in it often, you don't retain it.
radiowave 6y ago
I tend to apply much the same rule as emptyparadise - once conditional logic appears, it's time for a rethink. Part of my rationale for this is that I assume that I'm dealing with the thin end of the wedge - if there's enough complexity in the task to warrant conditionals, there's likely a whole bunch more (or there will be in future) that I haven't encountered or grasped yet. I'm thus willing to take the time-hit on rewriting in Python, because I'm betting that resulting code will be easier to adapt and easier to troubleshoot as this extra complexity gradually comes into view.
- spurgu 6y ago
  I just can't fathom this as I've never really had compatibility issues/quirks with good old #!/bin/bash. Only issues I've had have been with sed on Mac vs Linux. And I've written thousands of lines with lots of logic, generally breaking statements that become nested into separate functions. Sometimes some things become too complicated for Bash, like parsing JSON, in which case I prefer to break out and call a Python or Node script instead of wrapping my head around jq. But my scripts are also either for personal use, or for internal use (5 person team) at work, so I know who will be running the scripts, why, and where (which OS/versions). Suppose it's a bit like developing for Apple where you can remain aware of the use (and edge) cases.

dexen 6y ago

Awk is a viable replacement for Bash for scripting. A long established standard and installed on pretty much every POSIX system. It has more robust facilities for handling variables, arrays, regex search & substitution, etc.

usr1106 6y ago
awk is a full programming language. But it has gotten little attention in the last 20 years. If you expect that someone else can read and maintain your code with minimal learning I don't think it's the tool of choice.
- jrochkind1 6y ago
  But... neither is bash. I am sympathetic to the argument that awk is theoretically superior to bash in almost all ways (and it is of course nearly as available as bash). But... something still makes me reluctant to switch to using it in the cases I write a bash script (which I'm already trying to minimize, as is the OP). I'm not really sure why though. (Fun-for-mostly-me fact, my very first programming job ever was writing awk. It was a while ago. I certainly don't remember any awk now).
alexhutcheson 6y ago
Awk is great for snippets that are meant to be copy+pasted into a shell, and is the only reasonable option sometimes if you're stuck in a Busybox environment, but for almost everything else you'd be better off with Python. This previous comment of mine goes into a bit more detail: https://news.ycombinator.com/item?id=23242753 https://news.ycombinator.com/item?id=23242753
fomine3 6y ago
Agreed but I point that not all awks are gawk.

vlowther 6y ago

Or better yet, condition yourself to hit [ a second time and use Bash's built-in conditionals instead of test and [