6 ms·
I am always impressed how powerful and efficient shell scripts are but the syntax is just a killer. Is there a strong reason why all shells have this crazy synt
by maxxxxx 8y ago
I am always impressed how powerful and efficient shell scripts are but the syntax is just a killer. Is there a strong reason why all shells have this crazy syntax? Is it not possible to create a shell with a more expressive syntax like Java, C# or whatever?
- patrickmay 8y agoscsh (https://github.com/scheme/scsh https://github.com/scheme/scsh). Plus, it has the best Acknowledgements section of any technical documentation ever (https://scsh.net/docu/html/man.html https://scsh.net/docu/html/man.html).
- rashkov 8y agoWas glad to look up his CV and see that he's still around.. I'm not sure that 90s grumpy nerd humor has aged well into 2018, but I do appreciate the throwback to a different time
- psetq 8y agoAlternatively, Is there any reason today to not just use a language like Python/Ruby/JS for where shell languages would have been used?
- reificator 8y agoNumber one reason is availability. Do all of your current and future targets have the right version of Ruby installed? What if you have a mix of python 2.7 and 3.6?
- sethrin 8y agoI'm strongly considering making it a point to do just that: include Ruby in all the images I can, just to not have to use Bash. For one-liners it's fine, but I don't get ten lines into a Bash script before missing scripting-language features. However, if someone were to try to talk me out of what they consider to be an absurd notion, I would be prepared to take their criticism seriously.
- gkya 8y agoI have multiple ~10 line POSIX shell scripts in my dotfiles. I also have a bunch each of Perl and Python ones [1]. I'd rather not rewrite my shell scripts with neither of that languages, or Ruby (which I like and use a bit). When scripting, I find general-purpose languages are better when you have structured data (i.e. some JSON coming in) or you want to use libraries, and shell scripting better when you're wiring up multiple programs to work together, or starting up a program after setting up the environment for it. IIRC, the FreeBSD sh(1) man page was a good entry to writing POSIX compliant, thus quite portable shell scripts. [1] Compare the ones here: https://github.com/cadadr/configuration/tree/master/bin https://github.com/cadadr/configuration/tree/master/bin
- yjftsjthsd-h 8y agoIt really depends on target environment. At work? Sure, every single machine comes out of provisioning with Python 2.7 installed. For something that gets distributed to be used by other people? No way I'd trust anything but POSIX, and that's pushing it some days.
- oblio 8y agoTheir interaction with processes is very awkward compared to shells or even Tcl.
- frou_dh 8y agoUnlike (ba)sh, those languages are not a DSL for manipulating processes and files. (Ba)sh is more accurately framed as an ancient DSL than as a general-purpose language that has now been exposed as subpar.
- peterwwillis 8y agoA shell script is designed with file operators, environment variables, file handle redirection, process control, pipes/ipc, etc as first class primitives. It's just far easier to write a shell script if all you need to do is manipulate some executable programs.
- de_watcher 8y agoShell is like a LISP with less parentheses. And it's inside the text/files/executables environment.
- pas 8y agohttp://ammonite.io/#ScalaScripts http://ammonite.io/#ScalaScripts - this might look crazy (probably because it is), but it's there, it works, it has static typing, a nice scripting library ( http://ammonite.io/#Ammonite-Ops http://ammonite.io/#Ammonite-Ops ) it can be used as The Shell too. And now that JigSaw landed with JDK9 and with single file running coming in JDK11(?) and with precompiling bytecode to native, it becomes easier and easier to deploy good quality code as quick and fast scripts, tools, utilities, etc. Yes, of course, I still use bash, because it's there. But habits are easy to form and very hard to break. (Because why form re-form an old habit into a new?)
- reificator 8y agoI will give Microsoft credit that Powershell is a very pleasant to use shell language. There are a few quirks, but I encounter them a lot more rarely than in POSIX shells. That said, my brain does still think in POSIX scripting, so it always takes me a little while to get back in the swing of Powershell.
- maxxxxx 8y agoI gave up on PowerShell when I found out that the return value of a function is the console output. How s....d is that?
- reificator 8y agoYeah that's fair. You can work around it, but you really shouldn't have to.
- maxxxxx 8y agoThe common workaround is to pass in a dictionary that's then used for return values. Workable, but seriously? Is that the best they could do?
- reificator 8y agoThat is... not the way I generally work around it or see it. Normally you just use Write-Host when you need it and otherwise just collect return values inside the function by assigning them to variables. It's basically solved if you never call a command without storing the return value somewhere. Which is not optimal in a scripting language where you just want to get things done, but it's not the end of the world.
- jodrellblank 8y agoDo you have any examples? This doesn't sound familiar. Dictionaries are often used for return values, as they can be cast to PSCustomObject for a quick way to make objects for the pipeline. But in that use case they aren't passed in, and that's not anything to do with the `return` keyword or all output from all commands becoming function output.
- hyperpape 8y agoI've thought about this a lot, but assuming you want to be pleasant for interactive use, you're subject to a tremendous number of constraints. Just to start with: rm foo bar rm -rf * must be invocations of the rm command with the appropriate arguments. You need bare strings, you need variable interpolation, you need redirects. I think there's some room to break a few idioms and make something nicer, but it really is a hard problem. The choice to break things has to be based on how common an idiom is, how painful it is to not use it, and how much benefit you can get.
- laumars 8y agoI think you've hit the nail on the head there. I've been experimenting with writing my own shell and the original syntax was a lot more like Javascript but having to encapsulate parameters in parentheses, comma delimited and quotation was a massive pain in the arse. So I dropped the requirement for quotation marks, then made commas optional, then I dropped using periods to pipe (ala methods in OOP) and then eventually ended up rewriting the entire parser to follow a more traditional POSIX syntax. As it turns out, as ugly as Bash et al is, it actually makes a lot of sense for writing one liners in an interactive shell (as you stated yourself) So where I decided to deviate was areas I felt could be enhanced without breaking the POSIX syntax too significantly: * shell configuration, * error handling (something traditional shells are appallingly bad at) * and support for complex data structures (eg so you can grep through items in a minified JSON array as smartly as you can with a traditional stream of lines. I always respected the power of POSIX shells even before embarking on my pet project. However I never quite appreciated just how sane it's ugly syntax was until I attempted to replace it with something more readable. I mean sure there are still some specific areas I really don't agree with but that can always be argued as personal preference. As an aside note, one thing I didn't quite appreciate until writing my shell is just how much shells have in common with functional programming. Yes I know it's a far cry from LISP machines; but if you take away the variable interpolation and environmental variables then you're left with a functional pipeline that take something from STDIN and write it to STDOUT and STDERR and does so in a multi-threaded, concurrent, workflow by default. Weirdly this still makes POSIX shells more efficient for some types of data processing than writing a monolithic routine in a more powerful (or should that be "more descriptive"?) programming language. So there is some surprising elegance amongst all that ugliness.
- jolmg 8y agoThe fish shell was primarily created to be a shell language with a nicer syntax. I tried it for a few months a few years ago, and while the syntax is nicer, it misses a lot, and I mean a lot, of features. Adding those features would have, of course, complicated the syntax and gotten us back to something like zsh, so... I've been back on zsh ever since. Really, the only complaint I could make is why chose something silly like `fi` or `esac` to end compound statements, but such an issue is too superficial to abandon everything else that comes with these shells.
- cyunem 8y agoOut of curiosity, what features were you missing in fish? I've been using it for a while and haven't noticed anything, but I am probably not in need of heavy on shell scripting. I've heard of plenty of people choosing zsh instead of fish, but I started on fish and never felt the need to switch, so I'm kind of wondering what I might be missing out on.
- jolmg 8y agoThis could get very long, but to keep it short: Substitution via `=()`, `<()`, and `>()`. `=()` save's a process output in a temporary file that I don't have to mess with explicitly and substitutes itself with the path. An example of use is `viewnior =(maim -s)` which shows in an image viewer a window that I select; `diff -u <(...) <(...)` or `cmp`, or `comm` instead of `diff` compares the output of multiple processes without having to save them to files; `tee >(...) >(...) | ...` feeds the output of one command to multiple commands without needing to save files Subshells. I've used them interactively, and I prefer to not have troubles with quoting doing `fish -c '...'` in my commands. Extended globbing. It's far more concise and less error prone than using a combination of `find` and text processing tools. Dynamic directories. ~some_project/ for me expands to ~/work-for/client/some_client/some_project/ and works great with completion. What attracted me the most to fish was the ability to work with multi-line commands as a single command in the history. However, I've seen that zsh has better support for this. What killed fish for me back then was that their pipes were fake back then. This is something that they've since fixed, but back then, instead of running all commands at the same time with their inputs and outputs linked, I think fish would run the first command, save it to a file and then ran the second command with that file as input. I mean, the behaviour that I saw was that I wouldn't see any output until the first command finished, and some of the commands that I wanted to run took very long to finish or never were supposed to finish without me seeing some of the output first. I'm talking about watching file changes under a directory and manipulating the presentation of the output of the file watching command (e.g. `inotifywait -m ... | awk ...`) or looking for specific events in the system log as they happened (e.g. `journalctl -f | grep ...`) or looking for specific system calls of a never-ending running process (e.g. `strace -fe trace=file -p $pid | grep ...`). That made fish pipelines useless for me. Anyway I wrote a more lengthy post once describing the differences between bash and zsh that I liked: https://news.ycombinator.com/item?id=16963856 https://news.ycombinator.com/item?id=16963856 However, there must be more differences between fish and zsh. I can think of history syntax, right now. I can get an argument from any command I've ever typed based on a substring by doing !?substring?%. If I find myself calling two commands in succession multiple times, I can combine them in a single command without retyping them by doing `!-2; !!`. Then I only have 1 command to re-run next time instead of 2. If I typed a command and realize I want to include the last argument of the previous command, I can type `!$` to include it.
- kps 8y ago> Is there a strong reason why all shells have this crazy syntax? Stephen Bourne was a fan of Algol. The source to the shell is… interesting. Via a bunch of preprocessor macros, the (C) code looks like: IF !letter(*cp) THEN return(FALSE); ELSE WHILE *++cp DO IF !alphanum(*cp) THEN return(FALSE); FI OD FI return(TRUE);
- fanf2 8y agoThe bournegol macros are defined in https://minnie.tuhs.org//cgi-bin/utree.pl?file=V7/usr/src/cmd/sh/mac.h https://minnie.tuhs.org//cgi-bin/utree.pl?file=V7/usr/src/cm...
- chubot 8y agoI call it "failure by success". Unix is so incredibly successful that we are bound by decisions made 40 years ago. There is an unbroken chain from the Thompson shell circa 1972 to modern day bash. The language wasn't designed; it was accreted over decades. Looking at the difficulty of the Python 2 -> 3 transition might give you an idea of how hard it is to make incompatible changes, so that's what we're stuck with. It's a little bit like asking why C has so many warts -- e.g. the insecure standard library, "holes" in the type system like silently converting pointers to bools. But it's easier to "clean up" compiled languages than interpreted languages, and it was somewhat done with C89, C99, etc. You can have different compilation modes, and the user is more likely to fix things that a compile error flags. With something like shell, you don't have a good chance to surface errors and clean up corner cases. It just evolved over time until it was out of control. I'm working on fixing that with my Oil project: http://www.oilshell.org/blog/2018/01/28.html http://www.oilshell.org/blog/2018/01/28.html
- 3rdAccount 8y agoWe have a little hubris too. I spent years talking about how dumb Perl was. Then I learned Awk, Bash, and Unix and saw that Perl was a natural and evolutionary powerup with a massive script archive. Perl wasn't dumb, I just didn't understand the design decisions. Bash and many dynamic languages are similar. On another note, when I use Java/C# (languages I have little experience in), I always can't help but notice the lack of expressivity. I mean it takes 1/2 page of code to read a text file and print all lines that say "hello". In the Linux shell this could be: cat filename | grep -i "hello" Someone else will probably point out a better way. It is also only a couple of lines of Python.
- flukus 8y ago> Someone else will probably point out a better way. It is also only a couple of lines of Python. grep -i "hello" filename # ;) > I always can't help but notice the lack of expressivity. I mean it takes 1/2 page of code to read a text file and print all lines that say "hello" I think you hit the nail on the head there. I've used just about every build system known to man and we're currently using cake at work, which is uses c# as a scripting language. Everything is 10 times more verbose than the equivalent shell/makefile would be. Things like concatenating sql files are 30 lines instead of code instead of one liners.
- kccqzy 8y agoIt's just grep -i hello filename
- deleted 8y ago[deleted]
- 3rdAccount 8y agoHaha, thanks! I always forget :)
- adrianN 8y agoIt takes half a page because you get to choose things like buffering. Error handling is also much nicer.
- enriquto 8y ago> (...) but the syntax is just a killer What's so bad about the syntax? I find it rather beautiful. Can you give a (simple) example where you find the syntax ugly, and how would you prefer it to be?
- cup-of-tea 8y ago> more expressive syntax like Java, C# Really? Those are your best examples? The answer is Python in any case.
- coldtea 8y ago>Is there a strong reason why all shells have this crazy syntax For most things no. They could just as well have proper hashmap and array structures for example, instead of the clusterfuck they have. Or how about exceptions and try/catch (instead of traps and the like). Or the side effects of the bizarro [ command (it's a command, not just the syntax for an opening bracket). And tons of other things besides -- e.g. better argument parsing. Better ways to add auto-completion instead of the monstrocities of bash completion and the like.