9 ms·
I'm sorry to say to for me, shell scripting has lost. All my scripts are written in Python, which a thin shell script wrapper to launch it. Look at Oil. It tri
by pierrebai 6y ago
I'm sorry to say to for me, shell scripting has lost. All my scripts are written in Python, which a thin shell script wrapper to launch it.
Look at Oil. It tries hard to be bash compatible with optional buy-in to extension. Just use a superior language and give up on shell scripts, that's my advice.
- kbenson 6y agoI agree. Generally I find the approach to have a few tools that work well in their specific niches to be superior to having one tool for all jobs. I use a shell for shell stuff, I use a scripting language for most other stuff, and I might use a systems level compiled language (or a scripting language that calls into a compiled library) for more performance specific needs. If you're already within a specific area and only need to venture into the other for a minimal aspect of the current project, it can be useful to stick with what you're in, but you quickly reach the point where it's better to choose a better tool because of diminishing returns from using a tool for something it's not good for. Maybe the hammer in the hand is fine for prying the single board off fence or wall, but if you're going to be doing it to ten or twenty boards, maybe walking over to the shed to get the crowbar will save a lot of time and effort in the end.
- chubot 6y agothe approach to have a few tools that work well in their specific niches to be superior to having one tool for all jobs. Exactly, that is the point of shell. Shell scripts are meant to invoke Python or Ruby programs. sibling comment: https://news.ycombinator.com/item?id=24875932 https://news.ycombinator.com/item?id=24875932 earlier comment: https://news.ycombinator.com/item?id=24083764 https://news.ycombinator.com/item?id=24083764
- R0b0t1 6y agoThere is a better option and its name is Perl. Perl mixes the ins of shell scripts like easy argument and output passing with complex and easy to use control structures. You could use Python, but having been forced to use Perl extensively Perl is the superior choice for a complex shellscript-like workload. There's less boilerplate in Perl. There's a couple Python projects that come close like Fabric, but Perl was literally made for this type of workload.
- FreshFries 6y agoThere is nothing in the (IT) world I fear more then... Perl projects. The shear madness it will leash upon you when the dependencies fail is maddening. Never will I voluntarily touch anything written in Perl ever again in my live. I've seen the power of Perl, the beauty and elegance of it, but everything was ruined by the absolute shit show when it comes to its packet managers. /rant.
- johnisgood 6y agoI hate everything there is to hate about CPAN. I also hate how Perl libraries are handled. I have to have like 5 lines of code related to it in ~/.bashrc or ~/.bash_profile. Bleh. Perl libraries are always a problem when I want my program to run on other machines. No thank you.
- esaym 6y agoI find this to be a weird statement. I've never really fought with "cpan" but have had huge fights with node and python packages. The old school "cpan" command has not been recommended to be used for nearly a decade now. Maybe that is your issue? Everyone just uses cpan-minus: http://cpanmin.us/ http://cpanmin.us/ or "cpanm". It will install anything, if you have write access to the installed perl location, it will install globally, otherwise it installs packages into "~/perl5" then just do something like "export PERL5LIB=~/perl5/lib/perl5/" in your bashrc. Or maybe you are talking about having to compile code in certain packages? Alot of stuff using ssl has C code which links against openssl which is always a pain. Not sure what other issues you might be alluding to... There are some "packaging" tools to try to bundle and pin libraries, kind of like venv for python. I've never messed with them though. We just compile the latest perl binary and install all our needed cpan modules globally. Never worry about pinning since nothing is ever updated on cpan anymore...
- johnisgood 6y ago> The old school "cpan" command has not been recommended to be used for nearly a decade now. Maybe that is your issue? I did not know that it was not supposed to be used, so that might explain my issues, but at the same time it did work for me on my machine, but had a hard time running my Perl script on a VPS, because I had to set everything up again and for some reason I ran into some errors. I often ran into packages that failed to compile, too, yeah. > Never worry about pinning since nothing is ever updated on cpan anymore... Nothing makes it kind of sad. :/
- jonahx 6y agoThe author addresses this argument: > However, Python and Ruby aren't good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file system. But Python and Ruby have too much abstraction over these concepts, sometimes in the name of portability (e.g. to Windows). They hide what's really going on. From: http://www.oilshell.org/blog/2018/01/28.html#i-dont-understand-why-not-use-a-different-a-programming-language http://www.oilshell.org/blog/2018/01/28.html#i-dont-understa...
- dmix 6y agoSounds like we need a new standard!
- mikepurvis 6y agoEven something that should be as simple as piping output from one process to another in Python is a nightmare, and fraught with opportunities to shoot yourself in the foot with buffering deadlocks and other nonsense. The consequence of this is that many of my projects which require these overlapping domains (think stuff like build automation) ends up being the two interleaved— either an outer shell script that calls small Python programs (for parsing, URL fetching, and basically any error prone stuff where sane recovery/cleanup may be necessary), or it's a Python program that renders little templates of bash into strings and passes them to bash stdin. Or in some cases, it's CMake that's the outer orchestration piece, using configure_file to fill templates and then invoke them either at configure time (with execute_process) or at build time (with add_custom_command).
- bscphil 6y ago> Even something that should be as simple as piping output from one process to another in Python is a nightmare, and fraught with opportunities to shoot yourself in the foot with buffering deadlocks and other nonsense. This hits the nail on the head for me. What I'd like is something bash-like to handle running programs, pipes, tab-complete for file names, and so on, and something like Python syntax for control flow and strings (quoting / escaping), and access to something powerful like Python's standard math libraries (and ability to import other stuff like requests). I don't know how you'd roll all that into a single shell, but as close as you can get is what I'd like to see in a new shell. (I've briefly looked at Oil before, but it has seemed a bit complicated to merit trying to switch full time at this point. I'm definitely following it for future developments though.) Edit: just found Xonsh from another user's comment, which seems almost exactly what I'm looking for, if a bit hacked together at first glance. Going to try that out.
- swirepe 6y agoI'm curious what scripting tasks you do with python? I've found bash to be more than enough for everything I'd like to script, except for maybe stuff with heavy json processing (still doable with `jq`)
- ed25519FUUU 6y agoMany things that talk API and are writing/receiving json. Also I greatly prefer argparse. jq is nice but also a system dependency.
- mplewis 6y agoAnything with a non-trivial program flow, e.g. "do thing A, B, or C, and optionally mixin arguments D and E, with caching in between the shared steps." Lots of deploy scripts look like this.
- pmoriarty 6y agoI've actually started to transition my shell scripts to eLisp for better integration in to Emacs and eshell. As a Lisp, eLisp is not the greatest, but I'd still much rather use it than Python. I also don't want to sit and twiddle my thumbs while a Python script takes its sweet time in loading. Slow startup time is the kiss of death for most shell scripts.
- ed25519FUUU 6y agoPython startup time is slow for you?? It’s basically instantaneously on every machine I’ve ever used, unless there’s some seriously poorly written imports.
- jhayward 6y agoYou find Emacs to have a faster startup time than Python?
- ywei3410 6y agoI'm not the original poster, but judging from the answer it looks as though they'd already have an Emacs session open and can execute it without any extra startup-time; in which case the answer is yes, because there's no startup time.
- moritonal 6y agoCross-platform Powershell (specifically with Azure though) has become oddly enjoyable.
- chubot 6y ago(author here) I do use Python. In fact I wrote something like 30K lines of Python for Oil. The question isn't shell OR Python. My shell scripts call Python scripts, many of white I wrote myself. That is working as intended. They also call C programs, C++ programs and R programs. And put JavaScript programs in various places. ---- I guess this is a huge misconception about shell that I have to write a blog post about. https://news.ycombinator.com/item?id=24083764 https://news.ycombinator.com/item?id=24083764 I'm often asked this about Oil [1]: Why do you want to write programs in shell? That's not the idea of shell. The idea is that I write programs in Python, JavaScript, R, and C++ regularly, and about 10 different DSLs (SQL, HTML, etc.) And I work on systems written by others, consisting of even more languages. I need a language to glue them together. A language to express build automation and describe deployed systems. Most big systems consist of more than one language. Shell is the best language for that, but it's also old and crufty, with poor implementations. When you program in shell, gcc, git, pip, npm, markdown, rsync, diff, perf, strace, etc. are part of your "standard library". ----- If you want some concrete examples, look in the Oil repo. There are dozens of shell scripts that invoke custom tools in Python, R, and C++. https://github.com/oilshell/oil/tree/master/benchmarks https://github.com/oilshell/oil/tree/master/benchmarks For example, to generate this report on how much we speed up Python by translating it to C++: https://www.oilshell.org/release/0.8.3/benchmarks.wwz/mycpp-examples/ https://www.oilshell.org/release/0.8.3/benchmarks.wwz/mycpp-... The tables are manipulated by R code, and shell/Python/CommonMark generates the HTML. ----- Another example is that the release page is essentially a huge shell script: https://www.oilshell.org/release/0.8.3/ https://www.oilshell.org/release/0.8.3/ This page and all the linked pages are generated by: https://github.com/oilshell/oil/blob/master/devtools/release.sh https://github.com/oilshell/oil/blob/master/devtools/release... ------ If you don't understand why shell should call Python and C, I recommend reading The Art of Unix Programming http://www.catb.org/~esr/writings/taoup/html/ http://www.catb.org/~esr/writings/taoup/html/ https://www.amazon.com/UNIX-Programming-Addison-Wesley-Professional-Computng/dp/0131429019 https://www.amazon.com/UNIX-Programming-Addison-Wesley-Profe...
- CameronNemo 6y ago>When you program in shell, gcc, git, pip, npm, markdown, rsync, diff, perf, strace, etc. are part of your "standard library". I don't think this is a good characterization at all... Shell does not have git functionality (for example) built in. That is a dependency just like it would be for a python or rust project.
- msla 6y agoPython has a REPL but it isn't sufficient to be a shell. Therefore, I can't write a program in the Python REPL just as I'm doing stuff normally and copy it into a file for further editing and polishing. I have to port whatever I did in the shell to whatever other language I'm using, which is more porting than I want to do for most shell scripts, especially ones which are mostly pipelines. Heck, Python isn't even a good language for one-liners. Not saying it should be, but it further demonstrates that I can't use Python live the way I can use zsh live.
- edgyquant 6y agoHow is python not a good language for one liners? One liners are literally the pythonic way.
- msla 6y agoBy one-liners I meant things like Perl one-liners: perl -pe 's{<b>([^<]+)</b>}{<em>\1</em>}g' < foo.html > out.html Python would never let you get away with treating it like an improved sed.
- kortex 6y agoHave you tried xonsh? It's lovely. If you have python, you can install xonsh. It's packaged in apt, brew, conda, and probably more. It's pretty seamless to flow from bashlike pipelines to pythonic imports and functions. Autocomplete out of the box is better than bash, comparable to fish or ipython. https://xon.sh/ https://xon.sh/
- SPBS 6y agoIf your shell scripts are self-contained pieces of logic, you can use any language to write it. If you want to leverage external programs available in the shell, shell scripting is still the best glue language.
- barumi 6y ago> I'm sorry to say to for me, shell scripting has lost. All my scripts are written in Python, which a thin shell script wrapper to launch it. I've seen, and still see, plenty of Python scripts launched by a shell script. 9 out of 10 times, Python just causes more problems than it solves, and problems that are trivial to solve with plain old shell scripts. The only reason Python "won" is because we see a constant inflow of rookie developers who so far did practically only used Python ever, and that's pretty much the only thing they know and the only tool of their trade. They see all problems as nails, and thus insist in using their little hammer all the time without even looking at the current infrastructure or even the toolbox. I have seen shell scripts launch Python scripts. I have seen powershell scripts launch Python scripts. I have seen npm launch Python scripts. Hell, I have seen Python scripts launch Python scripts by firing up a Python interpreter as a child process. Python did not won. Python outnumbered. It did so because people who don't know better happened to know Python. Don't let that be the reason why you make terrible technical decisions.
- dimator 6y agoThis is ridiculous. Python won scripting because it offers a sane way to do sequential, shell-ish things, without having to wade through "man bash" or searching stack overflow for the umpteenth time about the syntax to do something that should be trivial but is anything but. Saying that younger devs only know python is like a FORTRAN engineer in the 90s saying young devs only know java. No one needs to apologize for growing up learning better mature readable shit. The number of gotchas and tricky nonsense in bash could (and probably does) fill books (array indexing, string comparison, quoting, toggling 'set -e', many more). I don't doubt that there are clever grey beards that are wizards that know the arcana. That doesn't mean arcane should be what you build an engineering culture around.
- barumi 6y ago> This is ridiculous. Python won scripting because it offers a sane way to do sequential, shell-ish things, without having to wade through "man bash" or searching stack overflow for the umpteenth time about the syntax to do something that should be trivial but is anything but. You've inadvertently supported exactly my point. You just advocate for the lazy way out. You know Python, so that's what you use and nothing more. God forbid you check out a reference. I mean, your lazyness leads you to believe that having docs.python.com on speed dial for a dozen different major or backwards incompatible releases is ok, but oh God forbid you check up a single man page of a tool that exists since the beginning of time. Ridiculous, and all of this just because you believe you know Python, and that's all you have to offer. Shell scripts are standard, omnipresent, reliable, and readily available infrastructure. There's no way around it. There's no excuse, let alone laziness and refusal to learn, which in this field is outright incompetence. The only reason someone refuses to use shell scripts for this sort of job is dereliction of duty and outright incompetence, and that is not winning in anyone's books.