5 ms·

Glue Languages Make You a Better Coder

50 points by joshuacc 15y ago

lysol 15y ago
If shell were a brand new language, made in someone's basement, it would be immediately shat upon. But its ecology and ubiquity make it 'good enough'. But really, shell programming is awesome in its simplicity. Bash syntax is a bit clumsy but every commandline application ever written for unix is a new library, provided it spits out reasonably predictable output. If unix applications are hunks of oddly-shaped and designed metal, shell programming is the arc welder. (Yes, I know I'm blurring the line between bash, sh, zsh, and the like, but they serve the same purpose.)
- holman 15y ago
  I've been saying more and more that "I hate shell almost as much as I love it" for those same reasons you mentioned. The syntax is so horrible while at the same time it's so powerful and clear and... lovely. Maybe I'm just strange, but even as a Rubyist who tends to get pedantic about loving clear syntax, I'm starting to really dig shell.
  - richcollins 15y ago
    I'm not sure I'd say that Ruby has a clear syntax. It's better than its predecessor, Perl, but there is a reason everyone uses the same parser code when they start a new Ruby VM -- it's a complicated mess. You certainly can write Ruby with a subset of its syntax that isn't a mess, but no one does. See Scheme and Io for examples of clear, simple syntax.
    - aangjie 15y ago
      Agreed. I haven't learnt ruby properly just read some code trying to fix a bug, but am no fan of ruby syntax. Neither am I of shell, but then shell is ubiquitous.And don't even get me started on perl :-). Ofcourse, my favourite language(syntax-wise) is python so far.
- bitops 15y ago
  I'm waiting for all the Java-bashing Rubyists out there to discover csh. If you need to do "real programming" csh is pretty great and has a more "programmery" syntax than a lot of the others. But, YMMV.
  - jleader 15y ago
    Way back when I was young and first learning shell scripting, I was taught not to use csh for scripting. Is that no longer true? Have the problems described here http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ been fixed in newer versions of csh, or in tcsh?
    - bitops 15y ago
      Okay, I'll take the bait. When I ssh into our Solaris server, it drops me into csh by default. That makes me think someone out there believes it's good enough.
      - artsrc 15y ago
        That article is focused on writing scripts, rather than interactive use.
      - zdw 15y ago
        That's not the default (/bin/sh is on recent versions of Solaris, which is fairly primitive), so your sysadmin is probably the person who thinks it's "good enough". Nearly all Solaris systems I've worked on and set up have a sun freeware or CSW system set up on them with all the standard "linuxy" tools installed, so the change isn't that weird.
        bitops 15y ago
        This was on a Sun Fire X4500 box set up by a sysadmin who I know for sure is not a csh guy. Could be that it was different for some reason.
      - 15y ago
- Goladus 15y ago
  The thing is about the shell is that it can't really be a brand new language. The shell derives its power directly from the OS and the OS needs the shell to provide much of that power. They almost have to evolve together. Also, bash syntax seems a lot less clumsy once you grok unix stuff like pipes, exit codes, and argument parsing (which you should really do anyway before scripting unix in any language). The fundamental abstraction is the command + argument list, and many quirks clearly derive from the syntax trying to stay out of the way of executing commands and gathering the results.
  - 1amzave 15y ago
    > The shell derives its power directly from the OS Indeed. After using it for quite some time without really thinking about it much, I've only relatively recently come to the realization that the shell is, in many ways, simply a thin very UI layer wrapped around the kernel's syscall API, and I think the awesome utility of the shell speaks to the quality of the design of that API.
    - Dobbs 15y ago
      The shell's name is derived from the fact that it was a wrapper around the kernel.
scott_s 15y ago
I suppose it's also true that your day-to-day development language may be someone else's glue language. There's probably hardware people who consider C their glue language. I use Python for all of my scripts. The moment that I want to automate some system related task that requires any logic, I reach for Python. (I'd still write a bash "script" if the script was just a succession of commands.) I don't think this is a controversial position since Python is considered a cousin of Perl, and Perl was heavily inspired by shell scripts, awk and sed. I think of Perl and TCL as the original glue languages. If I'm writing a script from scratch, I just don't see any compelling reason to do it in bash. (Or whatever your shell of choice is.) Yes, I recognize that "Python may not be available" is a potential reason, but it's not compelling to me because it's available in the places I'll need to work.
- pnathan 15y ago
  Agreed. I've been up to my armpits in other peoples' SH at times. It has a long list of defects which will come around and bite you. I have taken the policy of moving any SH script I write to Perl (or Python) after it breaks about 45 lines (a page). I get good error checking, type checking, reliable escaping, and the ability to neatly abstract into modules. So far, it hasn't hit me in the face yet. :-)
  - mturmon 15y ago
    Of course everyone does what works for them. But I have to say that I think the 45 lines rule is not the best criterion. Lots of scripts which are of the form "move these files around, run this command on the files, mangle the output, send it to this command to summarize, send to another command to pretty-print" can be long in line count, but are really well-done in shell. Basically, operation sequencing, and/or line-by-line file processing. Shell makes it easy to ensure tempfiles are deleted, to catch signals, to automatically check error status of each and every command. As you note, sometimes shell is "contraindicated" (as they say in medicine). Anything needing data structures, or abstraction beyond simple functions, is not so good for shell.
  - pyre 15y ago
    > reliable escaping One thing that I'm surprised about is that there is a severe lack of easy shell escaping of strings in more modern scripting languages. There should just be a printf symbol/modifier for it. I should be able to: sprintf("command %S %S", "blah blah blah", "blah -blah") to get command blah\ blah\ blah blah\ \-blah Also, take a look at: https://github.com/bsandrow/python-shellshock https://github.com/bsandrow/python-shellshock Let me know if you can think of anything that is painful to do in Python, but easy to do on the shell.
msie 15y ago
This is even apparent in shell: rvm, a platform-agnostic Ruby installer, is substantially written in shell rather than Ruby. I hope so! :-)
- saraid216 15y ago
  That comment didn't make sense to me. Why would a script that installs Ruby be written in Ruby? Isn't that the very definition of the chicken-and-egg problem?
  - holman 15y ago
    I think rvm could have a lot more Ruby code in it- Wayne (the author) does a lot of things in shell that could feasibly be abstracted away in Ruby. I happen to really like that, although I think it goes against the grain of what most Rubyists think- others would probably do the bare minimum in shell and quickly jump into Ruby as soon as it was feasible.
  - msie 15y ago
    I was hoping that rvm would mostly be written in shell script precisely because of the chicken-and-egg problem.
sidman 15y ago
I think the blogs comments regarding javascript are less true then that regarding shell as javascript is becoming a core language no longer only active in the web "glue language" domain. In the web context within HTML it maybe but in terms of mobile development using tools such as titanium/appcelerator or developing things in node.js, this requires javascript not as just a glue language but as the core. In terms of shell, yeh, I think it still applies cause to this day i havn't seen anyone write a fully blown program completely in shell (thought that could just be me). What i noticed in all the places i have worked at, when the shell script/program starts to get to large its usually slowly migrated to perl or <insert persons fav scripting language here>.
crazydiamond 15y ago
I've written stuff in shell over many years (I don't claim to be an expert, just average) but its not always smooth esp when I upgrade or change OS's. Most shell programs will make calls to other commands. These commands do not have standard arguments across unixes. Take the OSX (BSD) date and linux's date. On one machine I may have installed coreutils with default_names, on another without. Sometimes, this forces us to make quick trips into perl or awk (to get yesterday or tomorrow). However, shell scripting has certainly made me more productive over the years. For most of my initial programming life (90's), I've generated boilerplate code using shell (often with awk).
eddardstark 15y ago
"Node is just more accessible in JavaScript than if it were written in Ruby." Why? I don't see anything special in JavaScript's syntax that makes it more accessible. Node's asynchronous callback style would be a perfect fit for Ruby's block syntax.
- jcarden 15y ago
  I'm curious about this statement as well. Any leads ?

50 points by joshuacc 15y ago

lysol 15y ago

If shell were a brand new language, made in someone's basement, it would be immediately shat upon. But its ecology and ubiquity make it 'good enough'. But really, shell programming is awesome in its simplicity. Bash syntax is a bit clumsy but every commandline application ever written for unix is a new library, provided it spits out reasonably predictable output. If unix applications are hunks of oddly-shaped and designed metal, shell programming is the arc welder. (Yes, I know I'm blurring the line between bash, sh, zsh, and the like, but they serve the same purpose.)

holman 15y ago
I've been saying more and more that "I hate shell almost as much as I love it" for those same reasons you mentioned. The syntax is so horrible while at the same time it's so powerful and clear and... lovely. Maybe I'm just strange, but even as a Rubyist who tends to get pedantic about loving clear syntax, I'm starting to really dig shell.
- richcollins 15y ago
  I'm not sure I'd say that Ruby has a clear syntax. It's better than its predecessor, Perl, but there is a reason everyone uses the same parser code when they start a new Ruby VM -- it's a complicated mess. You certainly can write Ruby with a subset of its syntax that isn't a mess, but no one does. See Scheme and Io for examples of clear, simple syntax.
  - aangjie 15y ago
    Agreed. I haven't learnt ruby properly just read some code trying to fix a bug, but am no fan of ruby syntax. Neither am I of shell, but then shell is ubiquitous.And don't even get me started on perl :-). Ofcourse, my favourite language(syntax-wise) is python so far.
bitops 15y ago
I'm waiting for all the Java-bashing Rubyists out there to discover csh. If you need to do "real programming" csh is pretty great and has a more "programmery" syntax than a lot of the others. But, YMMV.
- jleader 15y ago
  Way back when I was young and first learning shell scripting, I was taught not to use csh for scripting. Is that no longer true? Have the problems described here http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ been fixed in newer versions of csh, or in tcsh?
  - bitops 15y ago
    Okay, I'll take the bait. When I ssh into our Solaris server, it drops me into csh by default. That makes me think someone out there believes it's good enough.
    - artsrc 15y ago
      That article is focused on writing scripts, rather than interactive use.
    - zdw 15y ago
      That's not the default (/bin/sh is on recent versions of Solaris, which is fairly primitive), so your sysadmin is probably the person who thinks it's "good enough". Nearly all Solaris systems I've worked on and set up have a sun freeware or CSW system set up on them with all the standard "linuxy" tools installed, so the change isn't that weird.
      - bitops 15y ago
        This was on a Sun Fire X4500 box set up by a sysadmin who I know for sure is not a csh guy. Could be that it was different for some reason.
    - 15y ago
Goladus 15y ago
The thing is about the shell is that it can't really be a brand new language. The shell derives its power directly from the OS and the OS needs the shell to provide much of that power. They almost have to evolve together. Also, bash syntax seems a lot less clumsy once you grok unix stuff like pipes, exit codes, and argument parsing (which you should really do anyway before scripting unix in any language). The fundamental abstraction is the command + argument list, and many quirks clearly derive from the syntax trying to stay out of the way of executing commands and gathering the results.
- 1amzave 15y ago
  > The shell derives its power directly from the OS Indeed. After using it for quite some time without really thinking about it much, I've only relatively recently come to the realization that the shell is, in many ways, simply a thin very UI layer wrapped around the kernel's syscall API, and I think the awesome utility of the shell speaks to the quality of the design of that API.
  - Dobbs 15y ago
    The shell's name is derived from the fact that it was a wrapper around the kernel.

scott_s 15y ago

I suppose it's also true that your day-to-day development language may be someone else's glue language. There's probably hardware people who consider C their glue language. I use Python for all of my scripts. The moment that I want to automate some system related task that requires any logic, I reach for Python. (I'd still write a bash "script" if the script was just a succession of commands.) I don't think this is a controversial position since Python is considered a cousin of Perl, and Perl was heavily inspired by shell scripts, awk and sed. I think of Perl and TCL as the original glue languages. If I'm writing a script from scratch, I just don't see any compelling reason to do it in bash. (Or whatever your shell of choice is.) Yes, I recognize that "Python may not be available" is a potential reason, but it's not compelling to me because it's available in the places I'll need to work.

pnathan 15y ago
Agreed. I've been up to my armpits in other peoples' SH at times. It has a long list of defects which will come around and bite you. I have taken the policy of moving any SH script I write to Perl (or Python) after it breaks about 45 lines (a page). I get good error checking, type checking, reliable escaping, and the ability to neatly abstract into modules. So far, it hasn't hit me in the face yet. :-)
- mturmon 15y ago
  Of course everyone does what works for them. But I have to say that I think the 45 lines rule is not the best criterion. Lots of scripts which are of the form "move these files around, run this command on the files, mangle the output, send it to this command to summarize, send to another command to pretty-print" can be long in line count, but are really well-done in shell. Basically, operation sequencing, and/or line-by-line file processing. Shell makes it easy to ensure tempfiles are deleted, to catch signals, to automatically check error status of each and every command. As you note, sometimes shell is "contraindicated" (as they say in medicine). Anything needing data structures, or abstraction beyond simple functions, is not so good for shell.
- pyre 15y ago
  > reliable escaping One thing that I'm surprised about is that there is a severe lack of easy shell escaping of strings in more modern scripting languages. There should just be a printf symbol/modifier for it. I should be able to: sprintf("command %S %S", "blah blah blah", "blah -blah") to get command blah\ blah\ blah blah\ \-blah Also, take a look at: https://github.com/bsandrow/python-shellshock https://github.com/bsandrow/python-shellshock Let me know if you can think of anything that is painful to do in Python, but easy to do on the shell.

msie 15y ago

This is even apparent in shell: rvm, a platform-agnostic Ruby installer, is substantially written in shell rather than Ruby. I hope so! :-)

saraid216 15y ago
That comment didn't make sense to me. Why would a script that installs Ruby be written in Ruby? Isn't that the very definition of the chicken-and-egg problem?
- holman 15y ago
  I think rvm could have a lot more Ruby code in it- Wayne (the author) does a lot of things in shell that could feasibly be abstracted away in Ruby. I happen to really like that, although I think it goes against the grain of what most Rubyists think- others would probably do the bare minimum in shell and quickly jump into Ruby as soon as it was feasible.
- msie 15y ago
  I was hoping that rvm would mostly be written in shell script precisely because of the chicken-and-egg problem.

sidman 15y ago

I think the blogs comments regarding javascript are less true then that regarding shell as javascript is becoming a core language no longer only active in the web "glue language" domain. In the web context within HTML it maybe but in terms of mobile development using tools such as titanium/appcelerator or developing things in node.js, this requires javascript not as just a glue language but as the core. In terms of shell, yeh, I think it still applies cause to this day i havn't seen anyone write a fully blown program completely in shell (thought that could just be me). What i noticed in all the places i have worked at, when the shell script/program starts to get to large its usually slowly migrated to perl or <insert persons fav scripting language here>.

crazydiamond 15y ago

I've written stuff in shell over many years (I don't claim to be an expert, just average) but its not always smooth esp when I upgrade or change OS's. Most shell programs will make calls to other commands. These commands do not have standard arguments across unixes. Take the OSX (BSD) date and linux's date. On one machine I may have installed coreutils with default_names, on another without. Sometimes, this forces us to make quick trips into perl or awk (to get yesterday or tomorrow). However, shell scripting has certainly made me more productive over the years. For most of my initial programming life (90's), I've generated boilerplate code using shell (often with awk).

eddardstark 15y ago

"Node is just more accessible in JavaScript than if it were written in Ruby." Why? I don't see anything special in JavaScript's syntax that makes it more accessible. Node's asynchronous callback style would be a perfect fit for Ruby's block syntax.

jcarden 15y ago
I'm curious about this statement as well. Any leads ?