10 ms·
The problem with the shell to me is that it started like a good idea, and then just stopped improving. Things that should be the bread and butter of scripting a
by dale_glass 5y ago
The problem with the shell to me is that it started like a good idea, and then just stopped improving. Things that should be the bread and butter of scripting are oddly arcane. Some examples.
How do you find where the script itself is? Because if the script comes with files, we want to find those when the script gets called from somewhere outside the current dir. Stack overflow suggests this as a first approach:
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
Then follows with 7 lines of arcane looking code containing a loop for a better solution. That's a ridiculous hoop to jump through for a common need.
Dealing with filenames? Hope you enjoy pain. Spaces, newlines and quotes are a horrible pain to deal with properly, and regularly cause trouble.
Dealing with correctly passing parameters? More of the same.
Dealing with long file lists? You run into the command line length limit.
Obtaining some incredibly common system parameter? Some insane combination of grep, awk and cut, probably.
Personally I find it amazing that people invented this thing, then spent decades tripping on the same quoting and filename handling issues and not finding that to be a good reason to fix it.
These days if it's more than 5 lines of code, or if arbitrary filenames are involved I go straight to Perl or Python, because the amount of completely preventable pain is drastically reduced.
- gnubison 5y ago> How do you find where the script itself is? No program can do this portably.
- chubot 5y agoI think you might be mixing two things up: it's impossible for a native program to find where it is, because argv[0] can be anything. But it's possible for an interpreter to tell the script it's interpreting where it lives, i.e. argv[1]. For example, Python does this in a pretty straightforward way.
- gnubison 5y agoYou're right, my bad.
- jakobnissen 5y agoNot to mention how dangerous it is to use. Accidentally type the > symbol? Enjoy overwriting whichever file came after in the command. Output to an existing file? Old file is gone. Accidentally put a space the wrong place? Gone. And somehow people think mastering this careless design is something to be proud of. Can anyone name any other popular piece of software that is that relaxed about irreversibly destroying your data?
- hpoe 5y agoWell "mastering [that] careless design" has made me farore capable than many of my peers at tasks that involve debugging network problems. I can slice and dice data files with ease and much more speed than any of my associates because the power the shell brings, I have gained a much deeper understanding of how am OS and computing works, what is going on under the hood, and consequently have reaped many advantages due to many hours spent mastering the shell. You are right it has some warts, but if a file is really that absolutely critical I make a .bak copy of it before I start my work. The shell has its limitations but so do all tools and the power it gives me is unmatched. As for other popular software there is plenty out there but other popular software out there destroys something far more valuable to me. My time.
- dTal 5y ago>You are right it has some warts, but if a file is really that absolutely critical I make a .bak copy of it before I start my work This seems to miss the point completely, unless you back up your entire hard disk before running every command. The whole problem is it's easy to nuke things you weren't intending to touch, accidentally. The user-hostility of the shell is not a required feature to achieve its power. All of the advantages you cite could be had without being a terrible footgun with human factors mired in 1970s computing culture.
- z3ncyberpunk 5y agoyou do realize Linux has file permissions which can prevent accidental deletion...
- thenoblesunfish 5y agoThe shell is a tool - one of many in your kit. I 100% agree that one should be quick to use Perl or Python for many tasks. But for some tasks, the shell is just the thing to pull out of your back pocket.
- meh99 5y agoThe problem I have with this is it’s just software. I am not a carpenter and context switching is expensive when it comes to thought work (probably because it’s still just typing, lacking the mechanics of swapping a hammer for a screwdriver). Anyway, the point is by now, 20+ years into an IT career, I should have at least ONE option that can handle these needs without flip flopping through an entire operating systems “tools”. The desktop metaphor is not the only real world model computing needs to get rid of. I want an interface that can do the job, not a million “tools”.
- Hello71 5y agoas I comment in basically every one of these shell articles, while Unix shell does kinda suck, the main problem is that the bad examples far, far outnumber the good ones, and so everybody programs in an overly complicated manner. SCRIPT_DIR="${0%/*}" works in practically all shells, as long as the script is actually invoked by path (if you do curl script | bash, you're SOL right off the bat). the same applies with "combination of grep, awk and cut". awk contains almost all the functionality of grep, sed, and cut, so any pipeline of awk plus one of the others is almost always unnecessarily complicated. it would be like complaining "in python, string handling is so hard. look, you have to do import re; for x in re.sub(...).match(5).split(' ')[3:-5]: lst.append(x)". of course if you make it overly complicated then it will be overly complicated. there are also too many footguns. spaces and newlines are trivial to handle in 95% of scripts, as long as you quote every expansion. as long as you never ever write cmd $file and always write cmd "$file", that solves virtually all whitespace problems. quotation marks in filenames are never interpreted in a special manner in Bourne-like shell unless you use eval, which has the same pitfalls as any other interpreted language (see python above).
- IncRnd 5y agoNow, invoke a script, containing that code, in the current directory, and you will see why his example was more complex.
- ori_b 5y agoSCRIPT_DIR="$(dirname $0)" It also works when you're not running bash.
- chubot 5y agoYes, those are all problems with bash and the state of the art. The funny thing is that there about a dozen ways to spell it, like $(dirname $(readlink -f $0)). Since the shell itself doesn't provide this functionality, people come up with a lot of workarounds. In Oil it will simply by $_this_dir, e.g. "source $_this_dir/mylib.sh" for relative imports. (Whenever a variable is silently mutated by the interpreter, akin to $?, Oil prefixes it with _). https://github.com/oilshell/oil/issues/587 https://github.com/oilshell/oil/issues/587 I solicit feedback on every Oil release: http://www.oilshell.org/blog/tags.html?tag=oil-release#oil-release http://www.oilshell.org/blog/tags.html?tag=oil-release#oil-r... so let me know what you think of the fixes: https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback Dealing with filenames is fixed in Oil: http://www.oilshell.org/blog/2021/04/simple-word-eval.html http://www.oilshell.org/blog/2021/04/simple-word-eval.html and with QSN: http://www.oilshell.org/blog/2020/10/osh-features.html#safe-processing-of-user-supplied-data-like-filenames http://www.oilshell.org/blog/2020/10/osh-features.html#safe-... Again, if you don't think this actually solves the problem, leave some feedback. Oil is the most bash-compatible shell by a mile, so you might need to use it someday :) (The theory is that in MOST situations you don't get to choose your language, just like you didn't choose C, C++, or shell. It's all the inertia of compatibility.) Oil has named parameters: http://www.oilshell.org/release/latest/doc/idioms.html#use-named-parameters-instead-of-1-2 http://www.oilshell.org/release/latest/doc/idioms.html#use-n... I know about the long file list problem, but I don't think I've ever run into it, mainly because I use xargs, which batches up commands correctly. If there is some other situation where it comes up, I'm interested. Insane combination of grep / awk / cut: Oil has eggex which compiles to ERE to help you write egrep and awk patterns: https://www.oilshell.org/release/0.8.11/doc/eggex.html https://www.oilshell.org/release/0.8.11/doc/eggex.html You get syntax errors when you write the pattern (at parse time), not when you run it. Crucially, we don't "rewrite" grep and awk. We just make them easier to use (optionally). There is a smooth upgrade path and you can retain your knowledge while forgetting about some sharp edges. > Personally I find it amazing that people invented this thing, then spent decades tripping on the same quoting and filename handling issues and not finding that to be a good reason to fix it. Yes I quoted David Korn complaining about the quoting problem in the early 90's, which was closer to Unix's invention than we are to the early 90's. https://www.oilshell.org/blog/2019/01/18.html#slogans-to-explain-the-project https://www.oilshell.org/blog/2019/01/18.html#slogans-to-exp... The reason we still have it is the inertia of compatibility, and the fact that nobody really owns shell. Even though people rightly complain about Google's web stewardship, HTML5 was a great improvement and cleanup. They paid people to fix HTML, and it worked to a large extent. Some food for thought: A Generation Lost in the Bazaar https://queue.acm.org/detail.cfm?id=2349257 https://queue.acm.org/detail.cfm?id=2349257
- fidesomnes 5y ago> and then just stopped improving. Nope. Not even once.
- enriquto 5y ago> How do you find where the script itself is? This question does not really make sense. There need not be any script, or there may be many at the same time. What exactly do you want? Imagine, for example, that you are piping the output of some commands into the shell.
- jjnoakes 5y agoIn that case, of course, the method of finding where the script lives would return something that indicates there is no script. But also in that case, the commands you are piping into the shell probably are not looking for the files they were bundled with, they are probably working on files you've explicitly specified, and so there is no problem with not finding the script's location in that case.
- coldtea 5y ago>There need not be any script Sure, but that is irrelevant. When there IS a script, you should be able to get its full path much easier, that is the complaint.
- enriquto 5y agoI agree that it may be possible to implement this feature in a nice form where it makes sense (for the people who need it). But it seems really strange to me and I cannot imagine a use for it. Conceptually, the shell only sees the contents of the script, why would it act differently according to where the script is? When you open a jpeg image, the jpeg library does not provide a facility to recover the directory where that file was found. For all you know, this may be an in-memory jpeg. Isn't the same with shell scripts?
- deleted 5y ago[deleted]
- coldtea 5y ago>Conceptually, the shell only sees the contents of the script, why would it act differently according to where the script is? Doesn't need to act differently in what it does. But it does need to report it, because e.g. you might want to bundle data/assets/config relative to the script, and want to be able to run it from wharever and have it still be able to find it.
- zxzax 5y agoDon't forget filenames that start with a dash... Usually you need to remember to put "--" in every single command invocation.
- usr1106 5y agoThat has to do absolute nothing with the shell, that's a Unix convention how commands parse their arguments.
- zxzax 5y agoI disagree, the shell is in the best situation to do something about it. Or at least it would be, if there was a consistent way for programs to report to the shell how their argument parsing works.
- NegativeLatency 5y agoYou might want to have a look at the fish shell or oil shell, some progress has been made
- jseban 5y agoDealing with files and directories? Best option. Dealing with calling multiple other tools written in different languages and/or network calls, and combine, store and do some basic operations on the data? Best option. It's a job control language after all. The pipes and redirects, and the instant access to the file system is also very powerful. Sure it could/should be improved massively, but I'd say it's still the best tool for a significant amount of pretty common use cases, it doesn't just suck at everything.
- jseban 5y agoTry writing this in another language: wget http://server{1..5}.com/file.txt >> /tmp/data.txt & wait && wc -l /tmp/data.txt There's some pretty serious stuff going on there
- actinium226 5y agoI'm surprised you didn't mention math. It's astounding to me how fragile it is to add 2 and 2 in the shell. And forget about floating point numbers. Anytime I need basic math in the shell I reach for 'python -c'
- kps 5y agoksh arithmetic includes floating point. Sadly bash made it to the extinguish step before embracing and extending that far. (Compound variables and getopts are the other big omissions.)
- techbio 5y agoIt is often easier to reach for my calculator, calculator app, or the address bar, to do arithmetic, than: echo "2 + 2" | bc -l
- racl101 5y ago> These days if it's more than 5 lines of code, or if arbitrary filenames are involved I go straight to Perl or Python Totally agreed. A shell script exceeding that's about 10 lines of code becomes so complex to read, interpret and improve that even though it may perform faster I would rather get the job done in a language that I can understand better when I come back to this 3 months later. Although ... on that note I don't think I'd ever choose Perl ha ha.
- warlog 5y agoOne does not... simply choose perl.
- pengaru 5y agoThe problem is we've historically limited ourselves to the lowest common denominator shell so our scripts worked everywhere without modifications. Hence everything has been stuck with the archaic limitations and oversights of posix shell or gnu bash as linux distros took over and largely standardized on it. There have been numerous alternative shells and some significantly improve on the situation, unfortunately they've been largely ignored. Default shell choice has proven very sticky. I do agree that it's practically impossible to write robust and safe scripts in bash/posix shell. You have to jump through many hoops if arbitrary user input is being handled. Frankly nobody should be writing generalized tools using such shells today, unless they have a very good reason to self-impose the constraint.
- oblio 5y agoYeah, but in the modern era, where almost all the ancient Unixes are for all practical purposes dead (you basically have to be an IBM consultant to use AIX, an HP consultant to use HP-UX, an Oracle consultant to use Solaris, *BSDs probably have 0.01% of Linux' usage rate), we live in a Linux mono culture. 99.99% of everything is internet connected and auto-updates. New distro versions come out every 2 years. There's no real reason why bash couldn't have been evolved into a sane language with sane parsing and with modern constructs. By sane, I mean... read this: https://www.oilshell.org/blog/ https://www.oilshell.org/blog/ At least these articles: https://www.oilshell.org/blog/2021/01/why-a-new-shell.html https://www.oilshell.org/blog/2021/01/why-a-new-shell.html (especially this one) https://www.oilshell.org/blog/2016/10/20.html https://www.oilshell.org/blog/2016/10/20.html https://www.oilshell.org/blog/2016/10/28.html https://www.oilshell.org/blog/2016/10/28.html https://www.oilshell.org/blog/2016/11/06.html https://www.oilshell.org/blog/2016/11/06.html If freaking Javascript could "use strict;" and turn a page, bash could have definitely have done the same. It's just that there's no real commercial interest and FSF doesn't have the manpower (or desire) to do this.
- JadeNB 5y ago> There's no real reason why bash couldn't have been evolved into a sane language with sane parsing and with modern constructs. By sane, I mean... read this: > https://www.oilshell.org/blog/ https://www.oilshell.org/blog/ I think the point is that, it did evolve, as your link shows—and there are lots of other sane alternative shells, too. The problem isn't that they don't exist; it's getting people to adopt them in sufficient numbers.
- GSGBen 5y agoThis makes me really hope to see more uptake of PowerShell in Linux (and performance improvements in PowerShell too). The above is just $PSScriptRoot.