6 ms·

The timeless beauty of shell scripts

43 points by brewski 16y ago

wglb 16y ago
While I am an inadvertent bash progammer--an astonishing number of lines of code end up in bash--I view the python challenge as more of an tutorial tool, and don't advocate replacing my shell scripts with python. I see this as a good way to learn python in the context of small problems to develop basic proficiency.
moe 16y ago
I'd agree with timeless. A pile of shell-scripts is the backbone of just about any deployment. And once you have them, they're not going away soon. Beauty? Not so much. Bash scripts are still the go-to solution because it's so quick to throw one together. Just paste what you have just tested on the CLI, add a few conditionals, done. Unfortunately the "done"-part more often than not drags out much further than we'd like. Suddenly there's a need to inspect the output of a process rather than just the return code. Suddenly the script should run from cron, but only one instance at a time please. Welcome to the not-so-beautiful world of lockfiles and the never-quite-complete shell environment in cron. This is the way our quick and beautiful 5-liners tend to turn into fragile 50-liners in short order. Little helpers like ftsh[1] can mitigate the mess somewhat. However, I for one try to use Python or Ruby for just about everything nowadays. It takes a bit longer to get off the ground that way - but it saves my slightly older self so much time and headache... [1] http://www.cse.nd.edu/~ccl/software/ftsh/ http://www.cse.nd.edu/~ccl/software/ftsh/
- Schmidt 16y ago
  I couldn't agree more, I've seen horrible >300 lines POSIX shell scripts used for deployments and thought "This is a stinking pile of shit".
- avar 16y ago
  The App::Cronjob Perl module is great for adding effortless locking to existing programs: @daily cronjob -j some-uid -c '/usr/bin/some-command.sh'
madair 16y ago
We evolve, or we die. Truism that. Sometimes it might take awhile, but its not less true.
geophile 16y ago
I really like the idea of connecting commands using pipes to do "one-off" commands. But piping strings is dumb and limits what you can reasonably do. It's too hard extracting what you want from the strings. At my last company, I needed a tool to interact with the nodes of a cluster, including the databases on each node. Putting this all together, I wrote a tool named Object Shell (http://geophile.com/osh http://geophile.com/osh), which is available under the GPL. It takes the idea of piping, but exposes Python language constructs on the command line. Python objects, not strings, are piped between commands. For example, I can write a command line to: execute a database query on each node; bring back the results with each row as a python tuple; combine the stream of tuples from each node into one stream; and then operate on the stream. Or I can write a command line to: get a list of process objects once a second; extract and transform properties of each process; dump the stream of data into a database. The Python objects have Python types, so I can operate directly on files, processes, numbers, times, etc. instead of strings representing those types.
- gaius 16y ago
  This is why PowerShell rocks - superficially they're text pipes like any shell, but under the hood it's passing COM objects around. Nice.
  - DrJokepu 16y ago
    Actually, those are CLR (.NET) objects, not COM objects. Sorry for nitpicking.
    - gaius 16y ago
      Heh, yep, old habits :-)
  - CrLf 16y ago
    For certain values of nice. While I find the PowerShell to be more powerful when I'm scripting stuff, I still prefer the unix shell mostly because it passes around and operates over text. Operating over text means you can compose a pipe progressively using what you see, without having to think about how it is structured internally. Operating over text makes the normal usage of the unix shell faster and more efficient than the PowerShell. That and the terser syntax. With the PowerShell I find myself always having to check the type of objects are being passed around and which attributes I should care about, while on unix I just "grep" and "awk" and I'm done. The PowerShell is more of a scripting language than a shell, while the unix shell is more of a shell than a scripting language. Although both of them are both those things.
spudlyo 16y ago
Most shell scripts are heaping mounds of undeclared external dependencies.
doki_pen 16y ago
Ruby is often just as terse as bash, and often simpler to understand. That said I still end up using bash, awk, sed, tr etc.. and don't revert to ruby until something gets complicated. Probably because bash is more portable.
- hernan7 16y ago
  The REPL-like development style is what does it for me... I always end up having to re-write it in Haskell.
- silentbicycle 16y ago
  Shell scripts are portable. Bash is specific to Linux, at best, and ... a bloated, hot mess. Programs that hardcode bash but only require basic bourne-shell features are a pet peeve of mine. "My favorite Linux distro has it out of the box" isn't a real standard.</bsd-porter>
  - avar 16y ago
    You can easily install bash on your BSD. But I get your point about writing POSIX compatible code. I'll declare a bash dependency in the shebang if I've only tested the script on bash. Requiring bash is better than having the script fail on some non-POSIX thing that I wasn't aware of. Linux distributions are getting a lot better in this department these days though, with dash being the default Debian/Ubuntu shell.
tkahnoski 16y ago
http://en.wikipedia.org/wiki/Domain-specific_language http://en.wikipedia.org/wiki/Domain-specific_language Unix shell scripts is listed as the first example.
houseabsolute 16y ago
$ tr a-z l-za-m $ tr -cd a-z Absent the comments, who the hell knows what either of these do? Certainly not the new guy on your team who has spent most of his life in the more popular environments.

43 points by brewski 16y ago

wglb 16y ago

While I am an inadvertent bash progammer--an astonishing number of lines of code end up in bash--I view the python challenge as more of an tutorial tool, and don't advocate replacing my shell scripts with python. I see this as a good way to learn python in the context of small problems to develop basic proficiency.

moe 16y ago

I'd agree with timeless. A pile of shell-scripts is the backbone of just about any deployment. And once you have them, they're not going away soon. Beauty? Not so much. Bash scripts are still the go-to solution because it's so quick to throw one together. Just paste what you have just tested on the CLI, add a few conditionals, done. Unfortunately the "done"-part more often than not drags out much further than we'd like. Suddenly there's a need to inspect the output of a process rather than just the return code. Suddenly the script should run from cron, but only one instance at a time please. Welcome to the not-so-beautiful world of lockfiles and the never-quite-complete shell environment in cron. This is the way our quick and beautiful 5-liners tend to turn into fragile 50-liners in short order. Little helpers like ftsh[1] can mitigate the mess somewhat. However, I for one try to use Python or Ruby for just about everything nowadays. It takes a bit longer to get off the ground that way - but it saves my slightly older self so much time and headache... [1] http://www.cse.nd.edu/~ccl/software/ftsh/ http://www.cse.nd.edu/~ccl/software/ftsh/

Schmidt 16y ago
I couldn't agree more, I've seen horrible >300 lines POSIX shell scripts used for deployments and thought "This is a stinking pile of shit".
avar 16y ago
The App::Cronjob Perl module is great for adding effortless locking to existing programs: @daily cronjob -j some-uid -c '/usr/bin/some-command.sh'

madair 16y ago

We evolve, or we die. Truism that. Sometimes it might take awhile, but its not less true.

geophile 16y ago

I really like the idea of connecting commands using pipes to do "one-off" commands. But piping strings is dumb and limits what you can reasonably do. It's too hard extracting what you want from the strings. At my last company, I needed a tool to interact with the nodes of a cluster, including the databases on each node. Putting this all together, I wrote a tool named Object Shell (http://geophile.com/osh http://geophile.com/osh), which is available under the GPL. It takes the idea of piping, but exposes Python language constructs on the command line. Python objects, not strings, are piped between commands. For example, I can write a command line to: execute a database query on each node; bring back the results with each row as a python tuple; combine the stream of tuples from each node into one stream; and then operate on the stream. Or I can write a command line to: get a list of process objects once a second; extract and transform properties of each process; dump the stream of data into a database. The Python objects have Python types, so I can operate directly on files, processes, numbers, times, etc. instead of strings representing those types.

gaius 16y ago
This is why PowerShell rocks - superficially they're text pipes like any shell, but under the hood it's passing COM objects around. Nice.
- DrJokepu 16y ago
  Actually, those are CLR (.NET) objects, not COM objects. Sorry for nitpicking.
  - gaius 16y ago
    Heh, yep, old habits :-)
- CrLf 16y ago
  For certain values of nice. While I find the PowerShell to be more powerful when I'm scripting stuff, I still prefer the unix shell mostly because it passes around and operates over text. Operating over text means you can compose a pipe progressively using what you see, without having to think about how it is structured internally. Operating over text makes the normal usage of the unix shell faster and more efficient than the PowerShell. That and the terser syntax. With the PowerShell I find myself always having to check the type of objects are being passed around and which attributes I should care about, while on unix I just "grep" and "awk" and I'm done. The PowerShell is more of a scripting language than a shell, while the unix shell is more of a shell than a scripting language. Although both of them are both those things.

spudlyo 16y ago

Most shell scripts are heaping mounds of undeclared external dependencies.

doki_pen 16y ago

Ruby is often just as terse as bash, and often simpler to understand. That said I still end up using bash, awk, sed, tr etc.. and don't revert to ruby until something gets complicated. Probably because bash is more portable.

hernan7 16y ago
The REPL-like development style is what does it for me... I always end up having to re-write it in Haskell.
silentbicycle 16y ago
Shell scripts are portable. Bash is specific to Linux, at best, and ... a bloated, hot mess. Programs that hardcode bash but only require basic bourne-shell features are a pet peeve of mine. "My favorite Linux distro has it out of the box" isn't a real standard.</bsd-porter>
- avar 16y ago
  You can easily install bash on your BSD. But I get your point about writing POSIX compatible code. I'll declare a bash dependency in the shebang if I've only tested the script on bash. Requiring bash is better than having the script fail on some non-POSIX thing that I wasn't aware of. Linux distributions are getting a lot better in this department these days though, with dash being the default Debian/Ubuntu shell.

tkahnoski 16y ago

http://en.wikipedia.org/wiki/Domain-specific_language http://en.wikipedia.org/wiki/Domain-specific_language Unix shell scripts is listed as the first example.

houseabsolute 16y ago

$ tr a-z l-za-m $ tr -cd a-z Absent the comments, who the hell knows what either of these do? Certainly not the new guy on your team who has spent most of his life in the more popular environments.