7 ms·
Shelljs – Portable Unix shell commands for Node.js
- statico 5y agoShellJS was a delight to use when we had to build a cross platform Electron-based (sorry) desktop application for Mac and Windows. It replaced batch and shell scripts, async operations were easy thanks to NodeJS, and the scripts were maintainable. 8/10, would use again.
- nerdponx 5y agoI feel like Node being "async-first" is such an underrated feature, compared to its big competitors (Python, Ruby, Perl). I really don't some of the Javascript quirks, but Node is a great general-purpose scripting tool as a change of pace from Python. Edit: I know we're not supposed to talk about voting on HN, but really? -2 for saying "hey maybe NodeJS is actually kind of OK for some stuff" is wild.
- VWWHFSfQ 5y agoI don't think "async-first" is really all it's cracked up to be. It's great for concurrent I/O (webservers). Not great for writing run-of-the-mill sysadmin scripts like rotating log files, doing file backups, etc. Now instead of just knowing that your cron job is doing something one at a time, suddenly you have all these async things happening and you look at your process table and there's 40 node.js processes trying to compress every log file in the system all at once. And if you don't want it to do that then you have to start managing the concurrency yourself. I've never had to manage concurrency in a shell script beyond just making sure my crons don't run over each other if they take too long.
- nwienert 5y agoYou’re conflating concurrency and parallelism. Unless you’re explicitly forking the process, the async all happens in one process. Async is incredibly easy to manage in node, even the simplest linter handles telling you you’re forgetting to await if you are.
- colordrops 5y agoYou can do it all synchronously in node if you wish. The async gives another dimension to work with.
- hamburglar 5y agoExcept the first two commands I looked at the implementation for were implemented as sync IO calls (and one of them was “cat” which is implemented as a sync read of the entire file).
- canadianfella 5y agoYou’re missing a word. Try fixing that.
- an_opabinia 5y agoOne thing I wish Windows developers did more was bundle Busybox, a 1.6mb binary, to run shell scripts with bashisms, rather than use batch files, powershell or other alternatives. If Microsoft had only put all the PowerShell for Linux resources into Busybox. MSYS2 is also pretty comfy. But that audience is probably better served by running WSL2.
- VWWHFSfQ 5y agoThis reminds me of a web hosting/managed colocation company I worked for back in the early 2000s. They pretty much only had unix sysadmins so basically every windows box got cygwin and activestate perl because it was really the only thing these guys knew how to use. Nobody ever bothered to learn how to script a windows machine. PowerShell was still in its infancy. But nowadays I don't think it's really appropriate to not use the platform-native tools to administer the machine. Imagine installing windows powershell on debian because you don't know how to write unix shell scripts.
- an_opabinia 5y agoPowerShell is so bad though, absolutely zero people do that. PowerShell for Linux is a straight up butt of jokes at Microsoft for having zero DAU.
- Tabular-Iceberg 5y ago> PowerShell is so bad though Care to elaborate? I’m stuck with Windows for bean counting reasons, and though I resent it I feel I would benefit from actually learning to make full use of it. But maybe it’s not worth the trouble?
- kordlessagain 5y agoSay goodbye to gnarly shell scripts? Oh Nodejs you’re funny!
- azalemeth 5y agoTo my naïve mind this seems like a sure-fire way to make them infinitely worse...
- mrbungie 5y agoDoes anybody know a project doing something similar but in Python?
- deleted 5y ago[deleted]
- kasbah 5y ago`sh` might be what you are looking for. https://amoffat.github.io/sh/ https://amoffat.github.io/sh/
- rjmorris 5y agoI'm pretty sure sh just runs the commands in a subprocess using the shell, so it isn't like shelljs at all. From the linked page: "Note that these aren’t Python functions, these are running the binary commands on your system by dynamically resolving your $PATH, much like Bash does, and then wrapping the binary in a function. In this way, all the programs on your system are easily available to you from within Python."
- zbentley 5y agohttps://pypi.org/project/plumbum/ https://pypi.org/project/plumbum/
- sime2009 5y agoPython's built in `shutil` package generally does a good job of covering the kinds of tasks you would use shelljs for.
- danellis 5y agoIf you're going to invoke these operations from JavaScript, why on earth would you still want to control them using command-line style switches? Reproducing the functionality: fine, useful. Reproducing the interface: inelegant and probably error-prone.
- anonydsfsfs 5y agoAccording to the README, the goal of this project is to "eliminate your shell script's dependency on Unix while still keeping its familiar and powerful commands". Changing the interface would add an unnecessary learning curve, since the target audience is already familiar with the conventional command-line style switches.
- rjmorris 5y agoMy concern with this approach is that it gives the impression that you can use all the same switches that are available to the real shell commands, which isn't the case. For example, taking the first command documented on the README, cat, shelljs supports the -n switch only, but `man cat` shows about 10 switches for me. So although I agree that the learning curve is lower to get started, you do have to remember that the switches are limited and be sure to check the docs to see which of your shell command switches are supported.
- anonydsfsfs 5y agoThat's a valid concern, but personally I wouldn't expect a tool like this to support anything beyond the POSIX standard switches. For "cat", POSIX only specifies a single switch: "-u", which enables unbuffered output. I've never used that switch before, and it's a no-op in both GNU cat and shelljs.
- smilekzs 5y agoI know this has been around for quite a while but I feel obliged to point out one major use for this: Embedding portable one-liners in the `package.json` file invoked through `npm run` [1], because you cannot assume from which shell is `npm` called by users of your package. [1]: https://docs.npmjs.com/cli/v7/commands/npm-run-script https://docs.npmjs.com/cli/v7/commands/npm-run-script
- paulddraper 5y agoBut users don't need to run those, right? Just contributors. (Unless you have some complicated install to perform which is good to avoid for multiple reasons.)
- deleted 5y ago[deleted]
- contriban 5y agoContributors might span across OS's, so this tool might still make sense
- franga2000 5y agoI guess, but if contributors have to install nodejs already, they might as well install cygwin if they're on Windows. All other platforms that I can think of are POSIX shell-compliant.
- contriban 5y agoI don’t disagree, they should definitely just use WSL anyway for their own peace of mind. Most node tools nowadays seem to be written for macOS first, something I would have never thought would happen 10 years ago, given how popular Windows computers were “for real work”
- 0mp 5y agoThere is also Modernish, which is a library for writing portable shell scripts and it seems to support all kinds of different versions of shells across a variety of operating systems. It really takes care of all the quirks and bugs every platform has. https://github.com/modernish/modernish https://github.com/modernish/modernish
- suifbwish 5y agoIs it really that difficult to just update a database value to trigger action by a listening process that checks for that value on a short time interval? No extra libraries needed and the shell and web server world remain completely separate.
- nodesocket 5y agoI believe Google just released https://github.com/google/zx https://github.com/google/zx. I am looking to refactor some "gnarly" and unreadable bash scripts in JavaScript and wondering what everybody thinks is the best option? Biggest pain points in my opinion in pure bash is flow control, string manipulation, arrays, and looping.
- sprobertson 5y agoWith the big caveat that I've tried neither, I'd probably go for zx if flow control is your main pain point. It seems Shelljs is more about adding shell to your node script, while zx is about adding node to your shell script.
- freedomben 5y agoIt's been my experience that for soe reason otherwise good developers who use functions liberally to modularize and DRY their code in <language X> seem to throw good software engineering priniples out the window when it comes to Shell. You'd be amazed at how readable and un-gnarly most shell script become if you break them into functions. Control flow and loops become quite easy. String manipulation on the other hand, if awk and jq can handle it reasonably well I'll use that, but I also forgive people for using a better scripting language. I prefer Ruby but I think using the language of the project is a better choice.
- krono 5y agoOh you hit the nail on the head! I have written some really basic shell scripts, and I try to use common shell modules/commands whenever it makes sense. It's just so confusing sometimes. The main challenges I faced in the beginning where caused by MacOS. MacOS comes with an older version of Bash, its default shell ZSH comes with its own slight variations on how it expects you to do things, and the utilities that come with MacOS aren't Gnu so the available options/arguments are not what you would find online. Then there is so much conflicting information online! A lot of it is also ancient, as in newsgroup archives from the 90s. String manipulation. When I look up how to do some specific replacement again, I come across all these massive inline awk scripts. But then I remember I could also just use sed and regex, or in some cases even just standard variable tricks, and achieve the desired result with 90% fewer code. The bigger scripts I've written are a mess.. I think. I still haven't been able to figure out how to error exit out of the entire script from a function that was called within it... :)
- dstaley 5y agoI'm a fan of Shelljs. I used it to translate shell scripts to run on any platform that supports Node[1]. I particularly like how the translated version is essentially a more readable version of the original shell script[2]. [1] https://github.com/Kinto/kinto-http.js/blob/master/bin/dist-fx.ts https://github.com/Kinto/kinto-http.js/blob/master/bin/dist-... [2] https://github.com/Kinto/kinto-http.js/blame/6a4ba42ba13a91a5121be2771688c2cab3dd423d/package.json#L19 https://github.com/Kinto/kinto-http.js/blame/6a4ba42ba13a91a...
- rodorgas 5y agoSo this is basically to support Windows, right? Because every other popular OS is Unix.
- skrebbel 5y agoYep, that's a key use case. Another good use case is replacing bash/sh scripts. I know some times who have a rule that as soon as there's a conditional or a loop, you gotta switch from bash to a friendlier / less error-prone programming language. ShellJS makes node a pretty good option.
- contriban 5y agoYes, 5 years ago. Nowadays, if you use ShellJS to replicate Unix commands, it means your team is mostly *nix users and you should probably just ask the rest of them to just use WSL and avoid a huge amount of problems that _not_ using it brings.
- diveanon 5y agoif anyone is looking for a way to use shelljs exec with async await you can do something like this https://pastebin.com/zUBGL3Vp https://pastebin.com/zUBGL3Vp
- BiteCode_dev 5y agoFunnily, I was thinking about this the other day with Python: python is great for scripting, with the generators, string formatting and stlib modules like pathlib, datetime or argparse. But there are still stuff I use bash for because it's way more efficient for one liners and small utilities. One of the things is that calling something in bash autoprints, unless you pipe it. Could it be done in Python? Well, it turns out you can, if you do a little dirty hacking with __call__, __repr__, __iter__, __or__ and __del__, you get: https://0bin.net/paste/XeMvMuSh#45izWhAtQ5Bd14PDWoBqTJMvHZXayl-dTtCyQbmxGpU https://0bin.net/paste/XeMvMuSh#45izWhAtQ5Bd14PDWoBqTJMvHZXa... Which will let you do: -ls('/etc'), auto prints all the files in the dir, in the shell and in a program. - ls('/etc') | keep(r'\.conf$'), pipe the output of ls, pass it to keep, filter it, then auto print it (again, works in the shell AND in a py file) - res = ls('/etc') | keep(r'\.conf$'), store the generator in the variable, doesn't print, doesn't calculate, ready to be able to be read with a for loop This means we could create a lib with a very shell like experience in python for all those 5-lines-scripts that are still too good to pass bash for. We could also make a @main decorator to auto run a function if __name__ == "__main__", borrow the "with cd()" and "with temp()" syntax from the fabric lib. The need for bash would get smaller and smaller for me. I rarely develop on a linux box that doesn't have python, after all. Now a problem people often hit is: yes, but you have to provide the dependency to your script. ShellJS proves that despite that, it has a great value already. But the good news is, it's not a problem anymore thanks to a great features of Python 3: zipapps. With shiv (https://shiv.readthedocs.io/en/latest/ https://shiv.readthedocs.io/en/latest/), you can bundle your python script and all its dependencies (including our dirty shell lib) in a single pyz file, which then can run anywhere there is a modern python, which is everywhere I personally need it.
- Calzifer 5y agoThe great thing with old tools is that it had decades of testing and most even obscure bugs are already found and fixed. If I run 'cat' in my shell I know it will print/output the file content. If I run 'shx cat' I get an interpretation of my file content. One use of 'cat' is to concatenate multiple files into one. Works great with classic cat, not so much with Shelljs. $ echo -ne "\0xac" > test.dat $ cat test.dat | xxd 00000000: ac . $ shx cat test.dat | xxd 00000000: efbf bd ...