21 ms·

JavaScript for Shell Scripting

385 points by gigel82 5y ago

graderjs 5y ago
This is cool. To the front page you go...
- graderjs 5y ago
  But then it happened. Don't hate. I was upvoter no. 2 and I just knew this thing was gonna be on PAGE1. I know it's an "non substantial comment", but still! :P ;) xx
Diti 5y ago
Why is Javascript a « perfect choose » for shell scripts?
- abiro 5y ago
  I thought that was weird too. I much prefer Python to JS for scripting.
  - Mauricebranagh 5y ago
    Or even Perl - I have only done some scripts in Python to use beautiful soup, or for scripts that may be used by less experienced developers.
- goatlover 5y ago
  Javascript all the things.
  - artificial 5y ago
    When the only tool you have is a hammer :)
    - bryanrasmussen 5y ago
      You look like Thor!
      - Mauricebranagh 5y ago
        More like Jeremy Clarkson hitting thing with a hammer instead of the right tool for the job.
- giorgioz 5y ago
  JS is the most used language for web development both client and server side. It's easy to find fullstack repos with only JS code and a bunch of shell scripts to launch the tooling like webpack bundler and cypress e2e tester. By wrapping the tooling also in JS it's possible to take advantage of shared configuration written in JS and write reusable functions to share across scripts. Currently I have all my tooling scripts in bash (not yet in JS) and it's becoming hard to manage. Sharing functions and constant values across folders in shell bash script is very cumbersome.
  - EvilEy3 5y ago
    > and server side. Cool story.
- Shacklz 5y ago
  I had very strong anti-javascript opinions for the longest of times, and still feel that writing any non-weekend-project in JS is a mistake. That being said, I have grown very fond of writing "plumbing"-scripts in nodeJS. The syntax is so much more sane than bash, the documentation of nodeJS is actually pretty good in my opinion, and once nodeJS is installed and available, its super easy to mash something together to get working. For small scripts, the uglier parts of JS do not really matter, and while nodeJS does have some APIs that feel a bit odd to use at times, it's at least well-documented with examples/guides all over the web. Some folks use python for that exact purpose, but I always felt that getting something going takes me more effort than nodeJS (the fact that I'm just not fond of Python as a language might also have something to do with this). All in all... for folks still using bash, I really recommend to give JS a try.
  - dehrmann 5y ago
    I prefer Python for this because more OSes ship with it, it has a fairly rich standard library, and SREs are more comfortable with it, so you're more likely to find others using it as glue.
    - throw_m239339 5y ago
      > I prefer Python for this because more OSes ship with it, it has a fairly rich standard library, I mean nodejs =/= javascript, the problem is nodejs and some of the people who designed it who basically made it so that it would rely heavily on NPM, a commercial solution, thanks to the virtually non existing standard library. Now Microsoft virtually owns Nodejs since it owns NPM. Nodejs creator himself regretted this decision, publicly.
      - Shacklz 5y ago
        For what I called "plumbing"-scripts above, I rarely ever need any dependencies via npm. What nodeJS delivers out of the box suffices completely. Sure, doing things like http directly with node is a bit cumbersome and having a library that does some abstraction comes in handy if you're working on something that goes way beyond what you'd ever consider doing in bash. But especially for that very purpose of things where I used nodeJS instead of using bash, nodeJS always had everything I needed out-of-the-box, almost never required any dependencies. That being said, if you re-use code, plan to share scripts etc. with other people and all these things - using something like typescript (and possibly some testing-libraries) comes in very handy for maintainability, and there you can't get around npm. But for one-off-"plubming-scripts", nodeJS had all I ever needed.
- throw_m239339 5y ago
  > Why is Javascript a « perfect choose » for shell scripts? It isn't, that project looks like perl re-invented but worse, since it mixes Javascript via nodejs and shell script syntax. Shell scripts already support loops, functions, conditions, variables, ... it makes more sense to write a nodejs script that works as a regular shell process and include it in a shell script than the other way around...
- deleted 5y ago
  [deleted]
- kristopolous 5y ago
  This is purely performative and has no other benefit than to bow to a set of aesthetics. It's merely doctrinal orthodoxy full of needless and ceremonious frivolities for the adherents to chin stroke in approval. It is not, in any way, a reasonable way to do things, to exercise engineering, write maintainable or reliable code or otherwise accomplish tasks. It is just spectacle.
alpaca128 5y ago
Neat idea, but what's the point of async-await when you just put await before 100% of calls like in the first example? Now you've got more to type for no gain. I don't get this popularity of async-await, especially in JS where I find its combination of syntax and absence of pre-run checks overly confusing and error-prone. And this, seriously? await $`exit 1`
- jiofih 5y ago
  If you want to have the same code structure as this, but without await, you’re gonna implement your own queuing system (every call adds an operation to a queue). Many old tools operated this way, before ES6. The problem with that it doesn’t meld together with most third party libs, since they will have no knowledge of your queue and will just execute immediately or out of order.
- devit 5y ago
  Yeah, feels like they should use the Haskell IO monad "do" notation style where the await is implicit unless you use let. This would require the scripts to no longer be vanilla JavaScript, but it seems easy to extend sucrase/babel to do that transpiling.
  - efortis 5y ago
    In plain JS: pipe(a, b, c) is equivalent to a().then(b).then(c) function pipe(...functions) { (async () => { let accum; for (const fn of functions) accum = await fn(accum); }()); }
- eyelidlessness 5y ago
  All IO in JS is async, you either have a Promise-based API (which can be sugared with await) or callbacks. The only exceptions are Node APIs which are explicitly synchronous in their names and CommonJS require. All of which are either discouraged or being generationally phased out (CJS->ESM). That it extends to the $ function (anything preceding a backtick in JS is a “tagged literal” but also a function call) is just API consistency because it can’t know whether you’re calling exit versus something which actually performs IO. So it always returns a Promise.
chetangoti 5y ago
This is good, enabling people with Javascript proficiency write complex scripts IMO. There is also https://github.com/shelljs/shelljs https://github.com/shelljs/shelljs which is implemented on top of nodejs APIs.
- jeswin 5y ago
  If you like shelljs, then check out https://bashojs.org https://bashojs.org (mine). # Example, list all dev dependencies: cat package.json | basho --json 'Object.keys(x.devDependencies)'
  - Too 5y ago
    I think use case of shelljs and this zx is more for developers who need to perform file system and subprocess operations but rather stay away from bash or other footgun-ridden shells and instead write things in safe code with access to richer data types and more libraries. bashojs seem to start in the other end, add js statement-execution to shells and and yet another pipeline-language in the mix. More like AWK with js-syntax. Looks good as a "jq" on steroids but i wouldn't compare it to shelljs.
- niffydroid 5y ago
  I pretty much got rid of my bash scripts and just replaced them using shelljs. Makes it a lot easier and quicker to maintain, it means any developer can jump in fix it and add to it.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
szhu 5y ago
Compare to Deno, which also makes JS friendlier for shell scripting! https://deno.land/ https://deno.land/ Coming from Deno, the single biggest advantage I see here is the handy tag function $: await $`cat package.json | grep name` Hoping someone will write a Deno port of this.
- Raed667 5y ago
  It would be an interesting CS problem to properly type the return values of random bash commands.
  - orta 5y ago
    This was kinda explored at the start of April this year: https://github.com/microsoft/TypeScript/pull/43480 https://github.com/microsoft/TypeScript/pull/43480
- maga 5y ago
  That's rather trivial with Deno.run, here I just wrote a gist for that out of curiosity: https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a1d72 https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a...
  - japanuspus 5y ago
    Thank you for posting -- I had never heard of tagged template literals! Reading the docs [1], it appears there is a `raw` property which might make this even simpler? [1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#raw_strings https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
    - maga 5y ago
      `raw` would allow us to pass the input as is without escaping it using JS rules, but otherwise the code would remain the same, we still have to "stitch" both arrays together with the loops.
parhamn 5y ago
Curious. When will JS tooling ecosystem support executing typescript by stripping the type information? Are there any plans to add generic soft type annotations to the spec? It’d be great to write these in TS without changing the runtime tooling.
- campers 5y ago
  What about using ts-node? You can run typescript files directly with that. I use it for my scripting purposes.
  - parhamn 5y ago
    It’s a great tool. It wouldn’t work in this case would it? Also, IIRC it checks types, doesn’t just strip (sometimes I prefer to just run the thing and do my type checking at the IDE/build level). Also still leaves the multiple tools problem.
    - CGamesPlay 5y ago
      Well `tsc` is the program that "strips" types after checking them. To be clear what you're asking for is a JavaScript runtime that supports parsing types but ignoring any errors it finds? Is there any computer programing language that has this behavior?
      - goodside 5y ago
        Python, pretty much exactly.
      - demurgos 5y ago
        Babel can do this if you wish to only strip TS extensions without typechecking.
    - campers 5y ago
      You can use the --transpile-only option to skip type checking. I'm going start using that myself for scripts I know are already good! Just tried it and makes a good difference to startup time.
bronlund 5y ago
Jesus Christ. As if Node wasn't bad enough.
johndoe42377 5y ago
This is getting beyond rediculous . It is already crossed into embarrassment.
austinshea 5y ago
Do people actually find this more convenient than bash? Their own readme isn’t trying to suggest that it’s an improvement, just some sort of convenience.
- mewpmewp2 5y ago
  More productive for folks who are comfortable with JS, but not bash, which would be very common for front end folks.
  - ttt0 5y ago
    Will people crucify me for suggesting that maybe you should also learn a language that's not JavaScript?
    - mewpmewp2 5y ago
      I should, but not sure if learning bash specifically is best for productivity. Especially if I don't use it daily, I will forget syntax etc, will still have issues understanding what is going on quickly, more likely to introduce bugs when trying to improve something etc.
      - gen220 5y ago
        The shell is a long-term investment, and decades of programmers are happy to quietly testify to its continuing dividends. POSIX shells (and OS) are the common thread underlying all *nix-based software. Bash prioritizes terseness and backwards compatibility; which is occasionally contradictory with UX, but are valid prioritizations nonetheless. Also, learning something new gives you a more nuanced appreciation of what you already knew. It’s a mind-expanding experience, even if it’s occasionally frustrating. TLDR. I’d advocate for it. It’s OK if things break or fall apart on the road, this is the price of knowledge acquisition.
        mewpmewp2 5y ago
        Makes sense, but I'm still afraid if I don't use it daily I will tend to forget it. I have learned everything I know by doing it in practice, so personally unless I would take some huger type of side project to learn up bash I don't see how I could do it. And in the end if I'm working with other frontend devs, I'd also have to convince them to learn bash as opposed to just using JavaScript/TypeScript for scripting. Do you have any suggestions on how one should go about learning it? If I do some course right now, then in a year's time if I use it maybe on few occasions in a year, I won't remember it, and it will stop being practical again.
pwdisswordfish8 5y ago
I hope the $ interpolator at least performs escaping instead of blindly concatenating shell scripts? Ah, who am I kidding. https://github.com/google/zx/blob/5ba6b775c4c589ecf81a41dfc91a6a5df00449b9/index.mjs#L30-L41 https://github.com/google/zx/blob/5ba6b775c4c589ecf81a41dfc9... function substitute(arg) { if (arg instanceof ProcessOutput) { return arg.stdout.replace(/\n$/, '') } return arg } // […] let cmd = pieces[0], i = 0 for (; i < args.length; i++) cmd += substitute(args[i]) + pieces[i + 1] for (++i; i < pieces.length; i++) cmd += pieces[i] Sigh… yet another code injection vector. And to think the whole template literal syntax was specifically designed with this in mind. Have people not learned from 20 years of SQL injection?
- jiofih 5y ago
  are you bringing in user input from a web form into your shell scripts?
  - pwdisswordfish8 5y ago
    Who knows, maybe? Or maybe I just want to process file names with spaces in them? Maybe I don’t want to worry about apostrophes in people’s surnames?
    - tutfbhuf 5y ago
      How do you deal with it, when you are writing bash scripts?
      - DonHopkins 5y ago
        By immediately invoking Python and getting the hell out of bash.
        FranchuFranchu 5y ago
        Or xonsh
exdsq 5y ago
Honestly I’d rather use Bash than JavaScript. I’ve been using Go for scripting recently which has been nice.
- Ashanmaril 5y ago
  Do you compile binary(s) or just execute a `go run` and rely on the fast build times?
zmix 5y ago
Nashorn (Oracle's, now discontinued ECMAScript implementation for the JVM and Rhino's successor) can be used for Shell scripting, too, when started with the `-scripting` parameter. Here is a little overview[1]. I think, GraalVM's new ECMAScript implementation is compatible with both NodeJS and Nashorn, including the scripting features. [1]: https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/shell.html https://docs.oracle.com/javase/8/docs/technotes/guides/scrip...
- brabel 5y ago
  GraalVM's JS engine[1] is an implementation of ECMAScript 2021 and as such, it's a drop-in replacements for Node.js and npm. It also supports some Nashorn features for interop with Java, but for many things to work, you need to enable it explicitly, especially the compatibility mode flag (`-Dpolyglot.js.nashorn-compat=true`). See https://github.com/oracle/graaljs/blob/master/docs/user/NashornMigrationGuide.md https://github.com/oracle/graaljs/blob/master/docs/user/Nash... [1] https://github.com/oracle/graaljs https://github.com/oracle/graaljs
deleted 5y ago
[deleted]
c0l0 5y ago
This embodies so much of what is wrong with some JavaScript programmers' (that I've known personally) mindset that I find it hard to distinguish from satire...
- classified 5y ago
  Yep, the quest continues to rewrite every software there ever was in JS.
  - the_cat_kittles 5y ago
    nothing wrong with wanting to avoid the insane syntax of bash imo.
    - c0l0 5y ago
      But you don't get to have that! You get the "insane" syntax oh bash, encapsulated in a ridiculously verbose and frankly quite unintuitive JavaScript contraption that seems to provide little, if any, benefit. Also, Shell scripting is mostly about correctly and safely interacting with other CLI-based tools - which this new thing wrapping a shell (and badly at that, see other comments in this thread) won't help you get right, either.
      - kube-system 5y ago
        There are some projects which I've migrated from bash to js, in the middle of the migration, we had a mix of both. This project would have been really helpful to make that migration quicker. Of course, it wasn't ideal, but there's always a journey before the destination.
      - the_cat_kittles 5y ago
        i feel like you could avoid some of the kind of annoying syntax like array iteration and conditional syntax. would it be better to know that stuff 100%, and write it in pure bash? yeah. is it one of those todos that people like me never seem to get around to, and this is kind of a stop gap? maybe. it doesnt seem entirely useless
gitgud 5y ago
Would be good to have a function which outputs all stdout from a sub command in real time, rather than after it's finished. I generally use "spawn" from child_process instead of "exec" (which this tool uses), which can pipe output to the terminal as it happens. Great for creating build scripts.
- lobstrosity420 5y ago
  Yes, you could use node's streams to implement this. It would be really cool.
franciscop 5y ago
I was hoping for an article on how to use Node.js for normal scripting, since it's already pretty close to what it's shown in this library. I've written two libraries to help with scripting in Node.js: `files`: https://github.com/franciscop/files/ https://github.com/franciscop/files/ import { read, walk } from 'files'; // Find all of the readmes const readmes = await walk('demo') .filter(/\/readme\.md$/) // Works as expected! .map(read); console.log(readmes); // ['# files', '# sub-dir', ...] `atocha`: simplest cli runner (no escaping though!) https://github.com/franciscop/atocha/ https://github.com/franciscop/atocha/ import cmd from 'atocha'; // Any basic command will work console.log(await cmd('ls')); // Using a better Promise interface, see the lib `swear` console.log(await cmd('ls').split('\n')); // Can pipe commands as normal console.log(await cmd('sort record.txt | uniq')); Both of them are wrapped with Swear, a "promise extension" (totally compatible with native promises!) so that's why the first example works. You can build operations on top of the return of the promise, so that these two are equivalent: // Without `swear`: const list = await walk('demo'); const readmes = list.filter(item => /\/readme\.md$/.test(file)); const content = await Promise.all(readmes.map(read); // With `swear`: constn content = await walk('demo').filter(/\/readme\.md$/).map(read);
- hackerbrother 5y ago
  Deno is great for this also! (https://deno.land/ https://deno.land/) This short wrapper is great for doing asynchronous SSH commands. https://github.com/gpasq/deno-exec https://github.com/gpasq/deno-exec
ape4 5y ago
Would the $ conflict with jQuery. (Of course jQuery's main job is DOM manipulation which isn't needed here)
- lobstrosity420 5y ago
  You can load JQuery into a different global variable if you want. There are cases where you might want to parse a DOM in the context of shell scripts, such as scrapping some web page. Although JQuery wouldn't be the tool I'd personally reach for in this case.
- LocalPCGuy 5y ago
  This isn't a browser-based script, so no real fear of conflict. Plus, it's basically past time to retire jQuery anyways, outside of maintaining legacy apps.
ilyash 5y ago
What if I told you there is a modern programming language designed specifically for DevOps? https://ngs-lang.org/ https://ngs-lang.org/ Born out of frustration with bash and Python. and ... nope, never considered JS for that type of scripting.
rattray 5y ago
Honestly the more appealing aspect than the "await $`cmd`" syntax is that it imports a bunch of handy libs by default. For some reason there's just something that rubs me the wrong way about having a bunch of requires at the top of a bash script (or cluttering my package.json with handy tools I use once in a script). But you're gonna want things like (promisified) fs, os, chalk, readline, node-fetch, etc quite a lot of the time. Definitely wish they'd included shelljs though; almost strange not to. I hope they add (or already have) syntax highlighting as .mjs without the file extension (just the shebang) for GitHub, VSCode, etc.
- tln 5y ago
  Currently, your scripts have to use the .mjs syntax to work because zx uses import() to actually execute the script.
justsomeuser 5y ago
As soon as I hit an “if” branch in a shell script I move to JS, Python or Ruby
- dahfizz 5y ago
  Do you have node installed on all Linux boxes you interact with?
  - justsomeuser 5y ago
    I typically deploy my code in docker containers. Often bash-like scripts are for my dev machine. But if portability was an issue I would still probably move to something that outputs a single binary (Go, Rust). I just think bash is not a very good language compared to the alternatives.
- dfinninger 5y ago
  I think if's are generally alright. My personal bar is arrays. Anything beyond for x in a b c; ... is getting written in Python.
  - justsomeuser 5y ago
    It's not the "if" in particular, it's more of a signal that I am moving from a simple set of commands to something more complex with logic that will need objects, arrays and possibly a package manager.
- joncp 5y ago
  Well, the whole reason to choose bash in the first place is that it just works on every single machine I have access to.. Ruby and js aren't installed everywhere. As for python, I have to be willing deal with versioning hell and the whole virtualenv vs venv vs conda mess. Maybe C could work, but I have to make sure it's statically linked.
  - pwdisswordfish8 5y ago
cltsang 5y ago
Atwood's law is still accurate after more than a decade: "Any application that can be written in JavaScript, will eventually be written in JavaScript." - Jeff Atwood source: https://blog.codinghorror.com/the-principle-of-least-power/ https://blog.codinghorror.com/the-principle-of-least-power/
christiansakai 5y ago
Lol. Anything that can be written in JavaScript will be written in JavaScript.
- chaostheory 5y ago
  It seems like Steve Yegge predicted it 10 years before it happened. Spot on.
- Bombthecat 5y ago
  It's getting really painful..
- devilduck 5y ago
  Except for Javascript itself, since it can't do that
stunt 5y ago
Javascript is just the new utility knife that all devs have in their pocket now mostly because you don't need to master it to build something with it, but also because it's acceptable to write improper JS code. PHP used to be the utility knife for a while for the same reasons. It was easy to learn, and run it. And it was ok to write bad PHP code. I remember most of the exploits and remote shells were written with PHP because it was the easiest language for hackers to learn.
- Toutouxc 5y ago
  JS as the utility knife? No thanks, I'll write my quick scripts in a language with an actual useful standard library. That would be Ruby for me, but I also accept Python and maybe dozens of others.
- nicbou 5y ago
  Python is a much better utility knife in my opinion, because it's used in more domains than just web development, and comes bundled with most operating systems.
  - devilduck 5y ago
    python is Way better than js but js people can't cope with other languages for some reason
cafard 5y ago
I occasionally have used JScript on Windows, mostly because I don't much like VBScript, and PowerShell want you to sign its scripts.
- vips7L 5y ago
  Just set your powershell execution policy to unrestricted? Thats the default on non-windows machines.
lucasmullens 5y ago
> JavaScript is a perfect choice I mean I love JS as much as the next person, but perfect? No way.
maxandersen 5y ago
Proper type safe Java scripts are nice too :) https://jbang.dev https://jbang.dev
vedantroy 5y ago
This reminds me of my `jsmv` tool, which I use for manipulating the file-system in place of Bash / find: https://github.com/vedantroy/jsmv https://github.com/vedantroy/jsmv
vedantroy 5y ago
Does anyone know why Bash has low usability? It seems like Bash has tons of footguns and unintuitive syntax. Is this just because the language grew organically?
ilaksh 5y ago
Maybe there could be a shorthand for the await keyword. Such as instead of await mycall() Maybe mycall()! Or something. Or perhaps a flag you could set at the top of the script that would transform every call into an await call.
sneak 5y ago
Why would you want a shell script to be async? All this means is that you're going to type await 47 times. This isn't a network server, I'm not sure why this is the right tool for this job. Edit: that parallel rsync example answers my question nicely. I should have read more carefully.
moocowtruck 5y ago
looks cool, but just want to drop this here https://book.babashka.org/ https://book.babashka.org/ https://github.com/babashka/babashka https://github.com/babashka/babashka
- iLemming 5y ago
  What's the point? Most people choose to suffer, even when you offer them something that requires learning the thing just once. No matter how nice the thing is.
juddlyon 5y ago
Better than bash but nothing beats Python for scripting IMO. I’m a mediocre programmer and search results are rich with Python examples for anything under the sun.
- louis-lau 5y ago
  To be fair, the same goes for javascript.
astrostl 5y ago
> Bash is great, but when it comes to writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice Awaiting citations on usually, convenient, and perfect. > await $`cat package.json | grep name` I mean, in the first line they're well into "I don't understand shells" territory with a superfluous cat pipe (you can just grep name package.json). This is seriously published under the google banner?
tusharsadhwani 5y ago
Honestly, i saw the syntax and thought I can do better, so I made it: https://github.com/tusharsadhwani/zxpy https://github.com/tusharsadhwani/zxpy
- cdaringe 5y ago
  he doesn't demonstrate it, but his allows for simple parallelism, of which the python version will be more arduous
deleted 5y ago
[deleted]
Ajnasz 5y ago
What is it good for? Why would you want to execute shell commands from JavaScript. If you already know the command, then why don't you just use the shell? Or why don't you just use JavaScript libraries/frameworks/whatevers if you can't make a shell script work but you know JavaScript?

385 points by gigel82 5y ago

graderjs 5y ago

This is cool. To the front page you go...

graderjs 5y ago
But then it happened. Don't hate. I was upvoter no. 2 and I just knew this thing was gonna be on PAGE1. I know it's an "non substantial comment", but still! :P ;) xx

Diti 5y ago

Why is Javascript a « perfect choose » for shell scripts?

abiro 5y ago
I thought that was weird too. I much prefer Python to JS for scripting.
- Mauricebranagh 5y ago
  Or even Perl - I have only done some scripts in Python to use beautiful soup, or for scripts that may be used by less experienced developers.
goatlover 5y ago
Javascript all the things.
- artificial 5y ago
  When the only tool you have is a hammer :)
  - bryanrasmussen 5y ago
    You look like Thor!
    - Mauricebranagh 5y ago
      More like Jeremy Clarkson hitting thing with a hammer instead of the right tool for the job.
giorgioz 5y ago
JS is the most used language for web development both client and server side. It's easy to find fullstack repos with only JS code and a bunch of shell scripts to launch the tooling like webpack bundler and cypress e2e tester. By wrapping the tooling also in JS it's possible to take advantage of shared configuration written in JS and write reusable functions to share across scripts. Currently I have all my tooling scripts in bash (not yet in JS) and it's becoming hard to manage. Sharing functions and constant values across folders in shell bash script is very cumbersome.
- EvilEy3 5y ago
  > and server side. Cool story.
Shacklz 5y ago
I had very strong anti-javascript opinions for the longest of times, and still feel that writing any non-weekend-project in JS is a mistake. That being said, I have grown very fond of writing "plumbing"-scripts in nodeJS. The syntax is so much more sane than bash, the documentation of nodeJS is actually pretty good in my opinion, and once nodeJS is installed and available, its super easy to mash something together to get working. For small scripts, the uglier parts of JS do not really matter, and while nodeJS does have some APIs that feel a bit odd to use at times, it's at least well-documented with examples/guides all over the web. Some folks use python for that exact purpose, but I always felt that getting something going takes me more effort than nodeJS (the fact that I'm just not fond of Python as a language might also have something to do with this). All in all... for folks still using bash, I really recommend to give JS a try.
- dehrmann 5y ago
  I prefer Python for this because more OSes ship with it, it has a fairly rich standard library, and SREs are more comfortable with it, so you're more likely to find others using it as glue.
  - throw_m239339 5y ago
    > I prefer Python for this because more OSes ship with it, it has a fairly rich standard library, I mean nodejs =/= javascript, the problem is nodejs and some of the people who designed it who basically made it so that it would rely heavily on NPM, a commercial solution, thanks to the virtually non existing standard library. Now Microsoft virtually owns Nodejs since it owns NPM. Nodejs creator himself regretted this decision, publicly.
    - Shacklz 5y ago
      For what I called "plumbing"-scripts above, I rarely ever need any dependencies via npm. What nodeJS delivers out of the box suffices completely. Sure, doing things like http directly with node is a bit cumbersome and having a library that does some abstraction comes in handy if you're working on something that goes way beyond what you'd ever consider doing in bash. But especially for that very purpose of things where I used nodeJS instead of using bash, nodeJS always had everything I needed out-of-the-box, almost never required any dependencies. That being said, if you re-use code, plan to share scripts etc. with other people and all these things - using something like typescript (and possibly some testing-libraries) comes in very handy for maintainability, and there you can't get around npm. But for one-off-"plubming-scripts", nodeJS had all I ever needed.
throw_m239339 5y ago
> Why is Javascript a « perfect choose » for shell scripts? It isn't, that project looks like perl re-invented but worse, since it mixes Javascript via nodejs and shell script syntax. Shell scripts already support loops, functions, conditions, variables, ... it makes more sense to write a nodejs script that works as a regular shell process and include it in a shell script than the other way around...
deleted 5y ago
[deleted]
kristopolous 5y ago
This is purely performative and has no other benefit than to bow to a set of aesthetics. It's merely doctrinal orthodoxy full of needless and ceremonious frivolities for the adherents to chin stroke in approval. It is not, in any way, a reasonable way to do things, to exercise engineering, write maintainable or reliable code or otherwise accomplish tasks. It is just spectacle.

alpaca128 5y ago

Neat idea, but what's the point of async-await when you just put await before 100% of calls like in the first example? Now you've got more to type for no gain. I don't get this popularity of async-await, especially in JS where I find its combination of syntax and absence of pre-run checks overly confusing and error-prone. And this, seriously? await $`exit 1`

jiofih 5y ago
If you want to have the same code structure as this, but without await, you’re gonna implement your own queuing system (every call adds an operation to a queue). Many old tools operated this way, before ES6. The problem with that it doesn’t meld together with most third party libs, since they will have no knowledge of your queue and will just execute immediately or out of order.
devit 5y ago
Yeah, feels like they should use the Haskell IO monad "do" notation style where the await is implicit unless you use let. This would require the scripts to no longer be vanilla JavaScript, but it seems easy to extend sucrase/babel to do that transpiling.
- efortis 5y ago
  In plain JS: pipe(a, b, c) is equivalent to a().then(b).then(c) function pipe(...functions) { (async () => { let accum; for (const fn of functions) accum = await fn(accum); }()); }
eyelidlessness 5y ago
All IO in JS is async, you either have a Promise-based API (which can be sugared with await) or callbacks. The only exceptions are Node APIs which are explicitly synchronous in their names and CommonJS require. All of which are either discouraged or being generationally phased out (CJS->ESM). That it extends to the $ function (anything preceding a backtick in JS is a “tagged literal” but also a function call) is just API consistency because it can’t know whether you’re calling exit versus something which actually performs IO. So it always returns a Promise.

chetangoti 5y ago

This is good, enabling people with Javascript proficiency write complex scripts IMO. There is also https://github.com/shelljs/shelljs https://github.com/shelljs/shelljs which is implemented on top of nodejs APIs.

jeswin 5y ago
If you like shelljs, then check out https://bashojs.org https://bashojs.org (mine). # Example, list all dev dependencies: cat package.json | basho --json 'Object.keys(x.devDependencies)'
- Too 5y ago
  I think use case of shelljs and this zx is more for developers who need to perform file system and subprocess operations but rather stay away from bash or other footgun-ridden shells and instead write things in safe code with access to richer data types and more libraries. bashojs seem to start in the other end, add js statement-execution to shells and and yet another pipeline-language in the mix. More like AWK with js-syntax. Looks good as a "jq" on steroids but i wouldn't compare it to shelljs.
niffydroid 5y ago
I pretty much got rid of my bash scripts and just replaced them using shelljs. Makes it a lot easier and quicker to maintain, it means any developer can jump in fix it and add to it.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

szhu 5y ago

Compare to Deno, which also makes JS friendlier for shell scripting! https://deno.land/ https://deno.land/ Coming from Deno, the single biggest advantage I see here is the handy tag function $: await $`cat package.json | grep name` Hoping someone will write a Deno port of this.

Raed667 5y ago
It would be an interesting CS problem to properly type the return values of random bash commands.
- orta 5y ago
  This was kinda explored at the start of April this year: https://github.com/microsoft/TypeScript/pull/43480 https://github.com/microsoft/TypeScript/pull/43480
maga 5y ago
That's rather trivial with Deno.run, here I just wrote a gist for that out of curiosity: https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a1d72 https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a...
- japanuspus 5y ago
  Thank you for posting -- I had never heard of tagged template literals! Reading the docs [1], it appears there is a `raw` property which might make this even simpler? [1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#raw_strings https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
  - maga 5y ago
    `raw` would allow us to pass the input as is without escaping it using JS rules, but otherwise the code would remain the same, we still have to "stitch" both arrays together with the loops.

parhamn 5y ago

Curious. When will JS tooling ecosystem support executing typescript by stripping the type information? Are there any plans to add generic soft type annotations to the spec? It’d be great to write these in TS without changing the runtime tooling.

campers 5y ago
What about using ts-node? You can run typescript files directly with that. I use it for my scripting purposes.
- parhamn 5y ago
  It’s a great tool. It wouldn’t work in this case would it? Also, IIRC it checks types, doesn’t just strip (sometimes I prefer to just run the thing and do my type checking at the IDE/build level). Also still leaves the multiple tools problem.
  - CGamesPlay 5y ago
    Well `tsc` is the program that "strips" types after checking them. To be clear what you're asking for is a JavaScript runtime that supports parsing types but ignoring any errors it finds? Is there any computer programing language that has this behavior?
    - goodside 5y ago
      Python, pretty much exactly.
    - demurgos 5y ago
      Babel can do this if you wish to only strip TS extensions without typechecking.
  - campers 5y ago
    You can use the --transpile-only option to skip type checking. I'm going start using that myself for scripts I know are already good! Just tried it and makes a good difference to startup time.

bronlund 5y ago

Jesus Christ. As if Node wasn't bad enough.

johndoe42377 5y ago

This is getting beyond rediculous . It is already crossed into embarrassment.

austinshea 5y ago

Do people actually find this more convenient than bash? Their own readme isn’t trying to suggest that it’s an improvement, just some sort of convenience.

mewpmewp2 5y ago
More productive for folks who are comfortable with JS, but not bash, which would be very common for front end folks.
- ttt0 5y ago
  Will people crucify me for suggesting that maybe you should also learn a language that's not JavaScript?
  - mewpmewp2 5y ago
    I should, but not sure if learning bash specifically is best for productivity. Especially if I don't use it daily, I will forget syntax etc, will still have issues understanding what is going on quickly, more likely to introduce bugs when trying to improve something etc.
    - gen220 5y ago
      The shell is a long-term investment, and decades of programmers are happy to quietly testify to its continuing dividends. POSIX shells (and OS) are the common thread underlying all *nix-based software. Bash prioritizes terseness and backwards compatibility; which is occasionally contradictory with UX, but are valid prioritizations nonetheless. Also, learning something new gives you a more nuanced appreciation of what you already knew. It’s a mind-expanding experience, even if it’s occasionally frustrating. TLDR. I’d advocate for it. It’s OK if things break or fall apart on the road, this is the price of knowledge acquisition.
      - mewpmewp2 5y ago
        Makes sense, but I'm still afraid if I don't use it daily I will tend to forget it. I have learned everything I know by doing it in practice, so personally unless I would take some huger type of side project to learn up bash I don't see how I could do it. And in the end if I'm working with other frontend devs, I'd also have to convince them to learn bash as opposed to just using JavaScript/TypeScript for scripting. Do you have any suggestions on how one should go about learning it? If I do some course right now, then in a year's time if I use it maybe on few occasions in a year, I won't remember it, and it will stop being practical again.

pwdisswordfish8 5y ago

I hope the $ interpolator at least performs escaping instead of blindly concatenating shell scripts? Ah, who am I kidding. https://github.com/google/zx/blob/5ba6b775c4c589ecf81a41dfc91a6a5df00449b9/index.mjs#L30-L41 https://github.com/google/zx/blob/5ba6b775c4c589ecf81a41dfc9... function substitute(arg) { if (arg instanceof ProcessOutput) { return arg.stdout.replace(/\n$/, '') } return arg } // […] let cmd = pieces[0], i = 0 for (; i < args.length; i++) cmd += substitute(args[i]) + pieces[i + 1] for (++i; i < pieces.length; i++) cmd += pieces[i] Sigh… yet another code injection vector. And to think the whole template literal syntax was specifically designed with this in mind. Have people not learned from 20 years of SQL injection?

jiofih 5y ago
are you bringing in user input from a web form into your shell scripts?
- pwdisswordfish8 5y ago
  Who knows, maybe? Or maybe I just want to process file names with spaces in them? Maybe I don’t want to worry about apostrophes in people’s surnames?
  - tutfbhuf 5y ago
    How do you deal with it, when you are writing bash scripts?
    - DonHopkins 5y ago
      By immediately invoking Python and getting the hell out of bash.
      - FranchuFranchu 5y ago
        Or xonsh

exdsq 5y ago

Honestly I’d rather use Bash than JavaScript. I’ve been using Go for scripting recently which has been nice.

Ashanmaril 5y ago
Do you compile binary(s) or just execute a `go run` and rely on the fast build times?

zmix 5y ago

Nashorn (Oracle's, now discontinued ECMAScript implementation for the JVM and Rhino's successor) can be used for Shell scripting, too, when started with the `-scripting` parameter. Here is a little overview[1]. I think, GraalVM's new ECMAScript implementation is compatible with both NodeJS and Nashorn, including the scripting features. [1]: https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/shell.html https://docs.oracle.com/javase/8/docs/technotes/guides/scrip...

brabel 5y ago
GraalVM's JS engine[1] is an implementation of ECMAScript 2021 and as such, it's a drop-in replacements for Node.js and npm. It also supports some Nashorn features for interop with Java, but for many things to work, you need to enable it explicitly, especially the compatibility mode flag (`-Dpolyglot.js.nashorn-compat=true`). See https://github.com/oracle/graaljs/blob/master/docs/user/NashornMigrationGuide.md https://github.com/oracle/graaljs/blob/master/docs/user/Nash... [1] https://github.com/oracle/graaljs https://github.com/oracle/graaljs

deleted 5y ago

[deleted]

c0l0 5y ago

This embodies so much of what is wrong with some JavaScript programmers' (that I've known personally) mindset that I find it hard to distinguish from satire...

classified 5y ago
Yep, the quest continues to rewrite every software there ever was in JS.
- the_cat_kittles 5y ago
  nothing wrong with wanting to avoid the insane syntax of bash imo.
  - c0l0 5y ago
    But you don't get to have that! You get the "insane" syntax oh bash, encapsulated in a ridiculously verbose and frankly quite unintuitive JavaScript contraption that seems to provide little, if any, benefit. Also, Shell scripting is mostly about correctly and safely interacting with other CLI-based tools - which this new thing wrapping a shell (and badly at that, see other comments in this thread) won't help you get right, either.
    - kube-system 5y ago
      There are some projects which I've migrated from bash to js, in the middle of the migration, we had a mix of both. This project would have been really helpful to make that migration quicker. Of course, it wasn't ideal, but there's always a journey before the destination.
    - the_cat_kittles 5y ago
      i feel like you could avoid some of the kind of annoying syntax like array iteration and conditional syntax. would it be better to know that stuff 100%, and write it in pure bash? yeah. is it one of those todos that people like me never seem to get around to, and this is kind of a stop gap? maybe. it doesnt seem entirely useless

gitgud 5y ago

Would be good to have a function which outputs all stdout from a sub command in real time, rather than after it's finished. I generally use "spawn" from child_process instead of "exec" (which this tool uses), which can pipe output to the terminal as it happens. Great for creating build scripts.

lobstrosity420 5y ago
Yes, you could use node's streams to implement this. It would be really cool.

franciscop 5y ago

I was hoping for an article on how to use Node.js for normal scripting, since it's already pretty close to what it's shown in this library. I've written two libraries to help with scripting in Node.js: `files`: https://github.com/franciscop/files/ https://github.com/franciscop/files/ import { read, walk } from 'files'; // Find all of the readmes const readmes = await walk('demo') .filter(/\/readme\.md$/) // Works as expected! .map(read); console.log(readmes); // ['# files', '# sub-dir', ...] `atocha`: simplest cli runner (no escaping though!) https://github.com/franciscop/atocha/ https://github.com/franciscop/atocha/ import cmd from 'atocha'; // Any basic command will work console.log(await cmd('ls')); // Using a better Promise interface, see the lib `swear` console.log(await cmd('ls').split('\n')); // Can pipe commands as normal console.log(await cmd('sort record.txt | uniq')); Both of them are wrapped with Swear, a "promise extension" (totally compatible with native promises!) so that's why the first example works. You can build operations on top of the return of the promise, so that these two are equivalent: // Without `swear`: const list = await walk('demo'); const readmes = list.filter(item => /\/readme\.md$/.test(file)); const content = await Promise.all(readmes.map(read); // With `swear`: constn content = await walk('demo').filter(/\/readme\.md$/).map(read);

hackerbrother 5y ago
Deno is great for this also! (https://deno.land/ https://deno.land/) This short wrapper is great for doing asynchronous SSH commands. https://github.com/gpasq/deno-exec https://github.com/gpasq/deno-exec

ape4 5y ago

Would the $ conflict with jQuery. (Of course jQuery's main job is DOM manipulation which isn't needed here)

lobstrosity420 5y ago
You can load JQuery into a different global variable if you want. There are cases where you might want to parse a DOM in the context of shell scripts, such as scrapping some web page. Although JQuery wouldn't be the tool I'd personally reach for in this case.
LocalPCGuy 5y ago
This isn't a browser-based script, so no real fear of conflict. Plus, it's basically past time to retire jQuery anyways, outside of maintaining legacy apps.

ilyash 5y ago

What if I told you there is a modern programming language designed specifically for DevOps? https://ngs-lang.org/ https://ngs-lang.org/ Born out of frustration with bash and Python. and ... nope, never considered JS for that type of scripting.

rattray 5y ago

Honestly the more appealing aspect than the "await $`cmd`" syntax is that it imports a bunch of handy libs by default. For some reason there's just something that rubs me the wrong way about having a bunch of requires at the top of a bash script (or cluttering my package.json with handy tools I use once in a script). But you're gonna want things like (promisified) fs, os, chalk, readline, node-fetch, etc quite a lot of the time. Definitely wish they'd included shelljs though; almost strange not to. I hope they add (or already have) syntax highlighting as .mjs without the file extension (just the shebang) for GitHub, VSCode, etc.

tln 5y ago
Currently, your scripts have to use the .mjs syntax to work because zx uses import() to actually execute the script.

justsomeuser 5y ago

As soon as I hit an “if” branch in a shell script I move to JS, Python or Ruby

dahfizz 5y ago
Do you have node installed on all Linux boxes you interact with?
- justsomeuser 5y ago
  I typically deploy my code in docker containers. Often bash-like scripts are for my dev machine. But if portability was an issue I would still probably move to something that outputs a single binary (Go, Rust). I just think bash is not a very good language compared to the alternatives.
dfinninger 5y ago
I think if's are generally alright. My personal bar is arrays. Anything beyond for x in a b c; ... is getting written in Python.
- justsomeuser 5y ago
  It's not the "if" in particular, it's more of a signal that I am moving from a simple set of commands to something more complex with logic that will need objects, arrays and possibly a package manager.
joncp 5y ago
Well, the whole reason to choose bash in the first place is that it just works on every single machine I have access to.. Ruby and js aren't installed everywhere. As for python, I have to be willing deal with versioning hell and the whole virtualenv vs venv vs conda mess. Maybe C could work, but I have to make sure it's statically linked.
- pwdisswordfish8 5y ago

cltsang 5y ago

Atwood's law is still accurate after more than a decade: "Any application that can be written in JavaScript, will eventually be written in JavaScript." - Jeff Atwood source: https://blog.codinghorror.com/the-principle-of-least-power/ https://blog.codinghorror.com/the-principle-of-least-power/

christiansakai 5y ago

Lol. Anything that can be written in JavaScript will be written in JavaScript.

chaostheory 5y ago
It seems like Steve Yegge predicted it 10 years before it happened. Spot on.
Bombthecat 5y ago
It's getting really painful..
devilduck 5y ago
Except for Javascript itself, since it can't do that

stunt 5y ago

Javascript is just the new utility knife that all devs have in their pocket now mostly because you don't need to master it to build something with it, but also because it's acceptable to write improper JS code. PHP used to be the utility knife for a while for the same reasons. It was easy to learn, and run it. And it was ok to write bad PHP code. I remember most of the exploits and remote shells were written with PHP because it was the easiest language for hackers to learn.

Toutouxc 5y ago
JS as the utility knife? No thanks, I'll write my quick scripts in a language with an actual useful standard library. That would be Ruby for me, but I also accept Python and maybe dozens of others.
nicbou 5y ago
Python is a much better utility knife in my opinion, because it's used in more domains than just web development, and comes bundled with most operating systems.
- devilduck 5y ago
  python is Way better than js but js people can't cope with other languages for some reason

cafard 5y ago

I occasionally have used JScript on Windows, mostly because I don't much like VBScript, and PowerShell want you to sign its scripts.

vips7L 5y ago
Just set your powershell execution policy to unrestricted? Thats the default on non-windows machines.

lucasmullens 5y ago

> JavaScript is a perfect choice I mean I love JS as much as the next person, but perfect? No way.

maxandersen 5y ago

Proper type safe Java scripts are nice too :) https://jbang.dev https://jbang.dev

vedantroy 5y ago

This reminds me of my `jsmv` tool, which I use for manipulating the file-system in place of Bash / find: https://github.com/vedantroy/jsmv https://github.com/vedantroy/jsmv

vedantroy 5y ago

Does anyone know why Bash has low usability? It seems like Bash has tons of footguns and unintuitive syntax. Is this just because the language grew organically?

ilaksh 5y ago

Maybe there could be a shorthand for the await keyword. Such as instead of await mycall() Maybe mycall()! Or something. Or perhaps a flag you could set at the top of the script that would transform every call into an await call.

sneak 5y ago

Why would you want a shell script to be async? All this means is that you're going to type await 47 times. This isn't a network server, I'm not sure why this is the right tool for this job. Edit: that parallel rsync example answers my question nicely. I should have read more carefully.

moocowtruck 5y ago

looks cool, but just want to drop this here https://book.babashka.org/ https://book.babashka.org/ https://github.com/babashka/babashka https://github.com/babashka/babashka

iLemming 5y ago
What's the point? Most people choose to suffer, even when you offer them something that requires learning the thing just once. No matter how nice the thing is.

juddlyon 5y ago

Better than bash but nothing beats Python for scripting IMO. I’m a mediocre programmer and search results are rich with Python examples for anything under the sun.

louis-lau 5y ago
To be fair, the same goes for javascript.

astrostl 5y ago

> Bash is great, but when it comes to writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice Awaiting citations on usually, convenient, and perfect. > await $`cat package.json | grep name` I mean, in the first line they're well into "I don't understand shells" territory with a superfluous cat pipe (you can just grep name package.json). This is seriously published under the google banner?

tusharsadhwani 5y ago

Honestly, i saw the syntax and thought I can do better, so I made it: https://github.com/tusharsadhwani/zxpy https://github.com/tusharsadhwani/zxpy

cdaringe 5y ago
he doesn't demonstrate it, but his allows for simple parallelism, of which the python version will be more arduous

deleted 5y ago

[deleted]

Ajnasz 5y ago

What is it good for? Why would you want to execute shell commands from JavaScript. If you already know the command, then why don't you just use the shell? Or why don't you just use JavaScript libraries/frameworks/whatevers if you can't make a shell script work but you know JavaScript?