17 ms·

Zx 3.0

189 points by medv 5y ago

ggambetta 5y ago
I thought this was going to be somehow related to the ZX Spectrum :(
miohtama 5y ago
Zx is good if JavaScript is only language you know. If you have ability to program or learn other programming languages, Python with packages like Shell and Plumbum might be better choices for shell scripting tasks that outgrow shell. - Python code with Shell is easier to read ana write - No nees to carry async/await keywords thru the code - Powerful plain text manipulation in the stdlib https://pypi.org/project/python-shell/ https://pypi.org/project/python-shell/ https://plumbum.readthedocs.io/en/latest/ https://plumbum.readthedocs.io/en/latest/
- gizdan 5y ago
  Wow, plumbum, despite it's unfortunate name, looks awesome. I will definitely try it next time I write a script.
  - philsnow 5y ago
    how is the name unfortunate? from the site: > Plumbum (Latin for lead, which was used to create pipes back in the day)
    - marcusramberg 5y ago
      And yet it sounds like a thicc ass.
      - mst 5y ago
        I confess to being quite amazed that you saw an innuendo somewhere I didn't.
    - dogma1138 5y ago
      Yes, but the name choice was quite intentional and probably giggle inducing. This is about as toilet humor as it can get.
      - FroshKiller 5y ago
        According to the Classical Latin pronunciation I learned in school, this is "ploom-boom" rather than "plum-bum."
      - bovermyer 5y ago
        I wonder if your comment says more about how you choose to see the name rather than any inherent meaning. Personally, I only see it as the old name for lead. Interesting to see how others are wired.
      - mst 5y ago
        I guess if you've never learned about the word 'plumbing' before?
    - gizdan 5y ago
      That's fair. My Latin is non-existent, and I clearly was too eager to look at example source instead of reading the description. I'll take my criticism and walk away.
- dogma1138 5y ago
  Powershell is also available for Linux these days, and well Powershell is awesome. Not sure how I feel about needing Node.JS to run shell scripts…
- alexghr 5y ago
  Maybe, but the main advantage of `zx` (at least for me) is that I don't have to manage another language environment. I can keep everything contained within Node :)
  - dogma1138 5y ago
    The problem is Node, especially all the packages that you’ll end up using under the hood, it’s seems quite risky especially if you don’t have super tight control over your Node SBOM. Would be a bit more interesting to me if it was a JS shell built from the ground up without the entire mess of the Node package dependency baggage.
    - alexghr 5y ago
      While I agree that Node's ecosystem is not the best, it's what we have in the frontend world. If you're working on a Node-based service or React app, you already have to deal with Node and its package ecosystem. Adding a couple of extra MB (or even KB, depending on package reuse) on top of your existing chain of dependencies is a tiny con to the huge benefit of not having to do logic in Bash :)
      - dogma1138 5y ago
        The problem is that you don’t always know what is actually being executed, shell scripting is risky as it is without the rabbit hole that is the Node dependencies system.
    - ptx 5y ago
      There's always WSH if you're on Windows and will settle for ECMAScript 3. No external dependencies or runtime needed.
    - arve0 5y ago
      zx has a small dependency tree, 12 packages marked as "@types" and 36 regular packages[0]. If you are fluent in Node, it should be easy to avoid further dependencies. 0: https://arve0.github.io/npm-download-size/#zx https://arve0.github.io/npm-download-size/#zx
- SahAssar 5y ago
  > Python code with Shell is easier to read and write That is pretty subjective
  - miohtama 5y ago
    Yes. I started programming in JavaScript in 1999 and Python in 2003, so it is my subjective opinion based on mastering the both languages. There are some factual and objevtive features making Python syntax more friendly, but it is a worth of a blog post to explore. And in the end there is no point, as JavaScript maximalists would not care.
- IshKebab 5y ago
  Maybe but Python is sloooow and its type annotation situation is pretty poor. I would recommend using Deno. It has the following advantages: * Uses Typescript which gives you a great static typing system. * Runs via V8 so it's about a million times faster than Python. * Single statically linked binary with no project setup files required (package.json/dependencies.txt) makes it about a million times easier to deploy than either Node or Python. Especially on Windows. * You can still use third party libraries even without a project file and you get IDE integration. It's the clear winner at this point. Someone has even made a version of zx for it: https://deno.land/x/zx_deno@1.2.2 https://deno.land/x/zx_deno@1.2.2
  - tcbasche 5y ago
    I’ve found Python is just fine performance-wise for CLI tools. What bad experiences have you had with it? I’ve mostly used click.
    - watermelon0 5y ago
      Python interpreter by itself is fast, it spawns in a few milliseconds, but it can slow to a crawl if you need libraries, and easily take a few seconds. However, situation is similar with nodejs.
    - IshKebab 5y ago
      Startup time is very slow. I just tested on my system and cold startup was 1 second (!) and hot 180ms. Vs 80ms and 64ms for Node. Also it obviously depends what your CLI tool is doing. Just because something is a CLI tool doesn't mean it doesn't do much stuff and therefore performance doesn't matter. For example Scons is a CLI tool. It's a build system written in Python. Everybody abandoned it because it was dog slow. Mercurial is a CLI tool. It's a VCS written in Python. Most people have abandoned it partly because it is slow (and partly because Github exists). You can read about Mercurial's troubles with Python performance here: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan > Performance is a significant pain point with Python. There are multiple facets to the performance problem: > * Startup overhead > * General performance overhead compared to native code > * GIL interfering with parallel execution > It takes several dozen milliseconds to start a Python interpreter and load the Mercurial Python modules. If you have many extensions loaded, it could take well over 100ms just to effectively get to a Mercurial command's main function. Reports of over 250ms are known. While the command itself may complete in mere milliseconds, Python overhead has already made hg seem non-instantaneous to end-users.
- wdroz 5y ago
  There is also xonsh[0] if you want an similar experience to zx to mix bash and python. [0] -- https://github.com/xonsh/xonsh https://github.com/xonsh/xonsh
maga 5y ago
And here is the same thing in Deno: https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a1d72 https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a...
- SahAssar 5y ago
  That doesn't quote/escape arguments and with zx you can do the shebang so you don't need the import but implementing that in deno shouldn't be too hard I think.
- searchableguy 5y ago
  https://deno.land/x/dzx@0.2.4 https://deno.land/x/dzx@0.2.4 Here is a more full fleded version.
TeMPOraL 5y ago
What does "DX" stand for in "tons of DX improvements"?
- DaGardner 5y ago
  Developer Experience I suppose
  - kleiba 5y ago
    Oh man, I'm getting too old for this shit...
    - marton78 5y ago
      It's like UX but for developers...
      - TeMPOraL 5y ago
        Which makes no sense if the developers are the users... But in the spirit of "there are only two business models - bundling and unbundling", I guess there are only two marketing tricks: use an old term for a new thing, and introduce a new term for an old thing...
        zeroxfe 5y ago
        Or more charitably, while the UX/DX distinction isn't very meaningful for this project, there are lots of products (e.g., payments) that target both (non-developer) end users and developers. It helps to be able to separate the personas.
        TeMPOraL 5y ago
        I don't think this holds. As the rule goes, there are only three cardinal numbers: zero, one, many. What are the chances you product has only two meaningful personas - "users" and "developers"? Take that payment system. People who buy things are obviously users, people who write CMS plugins for that system are obviously developers. But what about, say, analysts studying reports from that system? Accountants making sure the money flows where it should? Sysadmins keeping the backend components running? These are all users too. Where UX makes sense as a broad concept, giving a separate acronym to one small subset of potential users... doesn't make sense.
ilaksh 5y ago
I feel like the await keyword everywhere is a bit verbose. One might use something like the gpp preprocessor (hopefully the macros could be defined in a separate file): #!/usr/bin/env zx #define _ await $ #define __ await Promise.all #define _1 process.argv[1] #define _2 process.argv[1] #define _3 process.argv[1] _`cat package.json | grep name` let branch = _`git branch --show-current` _`dep deploy --branch=${branch}` __([ $`sleep 1; echo 1`, $`sleep 2; echo 2`, $`sleep 3; echo 3`, ]) let name = 'foo bar' _`mkdir /tmp/${_1}`
- williamdclt 5y ago
  I don't think it is very relevant to critique the Javascript syntax on a post about a specific library
  - masklinn 5y ago
    This is a node package, so using async/await would have been a specific design decision of the library.
  - ilaksh 5y ago
    I am not talking about JavaScript in general. I have been doing mainly JavaScript programming for the last 8 or 9 years or so (before that other languages like C# and C++ for many years). And I use those keywords all the time. For this specific use case, they are trying to replace shell scripts. So I thought maybe some people might be willing to consider abbreviations for the async and await keywords for this case.
- deleted 5y ago
  [deleted]
mikevm 5y ago
Ugh, typing `await` for every command is terrible. JavaScript's async-by-default model is clearly not a good fit for this use case.
- lucideer 5y ago
  Not sure what this comment means. Javascript has no "async-by-default" model. The author of this library has chosen to make the "$" function they provide async; this isn't a JS design feature or limitation, it's just the lib author's choice.
- lhnz 5y ago
  I wonder if they could use some kind of magic to create synchronous versions of all async APIs (e.g. https://giuseppegurgone.com/synchronizing-async-functions/ https://giuseppegurgone.com/synchronizing-async-functions/) Although this would be fighting the language...
- tgsovlerkhgsel 5y ago
  Requiring it for every command by default sucks, but on the other hand, the ability to easily make scripts async is a huge benefit. I've seen so many cases of processes being synchronous simply because doing it differently would require an unrealistic amount of custom parallelism code.
- Too 5y ago
  I think the idea is that the majority of the script will be written in pure JS, with only occasional calls out to subprocesses, so “every command” is far from every line. You frequently see the same mental mismatch when shell-scripters try to use Python, and complain how hard it is to pipe with POpen, instead of using the built in string processing features.
satyanash 5y ago
I personally find Ruby's built-in shell features more ergonomic: `cat package.json | grep name` branch = `git branch --show-current` `dep deploy --branch=#{branch}` if $?.success? [ "sleep 1; echo 1", "sleep 2; echo 2", "sleep 3; echo 3", ].map { |cmd| Thread.new { %x(cmd) } }.each(&:join) name = 'foo bar' `mkdir /tmp/#{name}` if $?.success?
- eurekin 5y ago
  Does this carry over to jruby as well?
- chubot 5y ago
  How do you do shell escaping of #{name} if it comes from user input?
  - pornel 5y ago
    The unnecessary "from user input" qualification is why spaces are kryptonite for most shell-based tools. It doesn't matter where input comes from. Input needs to be escaped for the context it's used in, period. Non-exploitable doesn't mean it's correct.
- jitl 5y ago
  What I don’t like about backticks in Ruby is that they “ignore” errors in commands you run. It’s up to the program author to remember to check $? for the last executed command’s exit status. And guess how many times the average Ruby script using this feature implements error handling? Usually it’s totally forgotten. To be safe, abstractions that make it easy to shell out must also: - escape all variable interpolations by default using “shellwords” or similar. - throw exceptions on error by default. Backticks in Ruby are very easy to use, but aren’t safe.
  - eatonphil 5y ago
    Ruby noob, but surely there's a way to metaprogram and change this to default to throw on non-zero exit?
underdeserver 5y ago
Very cool!
kjgkjhfkjf 5y ago
I'm having a hard time believing that Google uses this much internally. Is github.com/google just a place where Googlers can put personal projects to get additional exposure?
- tym0 5y ago
  The copyright of the thing Googlers build on their oswn time are usually owned by Google and so it ends up on the Google org if they want it on GitHub. They usually have a disclaimer like "this is not a official Google product..."
  - codewiz 5y ago
    This isn't correct: if a Googler writes some code from scratch, which is unrelated to their work, on their personal time, and using their personal laptop... then it's definitely not owned by Google. There could be gray areas where code ownership could be disputed, but in general your employer owns only the intellectual property it paid for. Source: I'm a Googler (until next month).
    - wokwokwok 5y ago
      Are you sure? Not in the “haha, yes of course”, in the “I’ve spoken to HR and have it writing” kind of way. I’ve heard this is not true from others, it is simply true in certain geographical regions where local laws override the contract you signed.
      - mrgriscom 5y ago
        "The provisions of this Agreement requiring disclosure and assignment of Inventions to Google do not apply to any invention that I have developed entirely on my own time without using any of Google Property (as defined below) or Google Confidential Information, except for those inventions that either (i) relate at the time of conception or reduction to practice of the invention to Google’s business, or actual or demonstrably anticipated research or development of Google or (ii) result from any work or services that I performed for Google."
cfeliped 5y ago
In my first job I gave a try at using Python for scripts. It was a terrible idea, as I now had 3 problems instead of 1: 1 - Writing scripts, plus: 2 - Installing Python in every host that needed to run those scripts 3 - Maintaining Python and the necessary dependencies up to date in each host / container. (2) and (3) are trivial when done just once in your own computer, but end up being a huge time sink for larger heterogeneous environments with (possibly) network and security barriers. Moreover, if you are using containers, installing Python will make images unecessarily large. Granted, I'm not a JS person, but I imagine one will have the same issues. Even though bash has its shortcomings, I really appreciate how low maintenance it is as long as you keep scripts small and sane.
- qalmakka 5y ago
  That's why Perl is my go-to choice for glue scripts. It's almost always installed by default, it has stayed consistent for decades without any breaking changes, its regex support is by far the best around, and its almost as good as the shell at gluing commands together.
  - nyanpasu64 5y ago
    I'm less likely to have Perl installed than Python on Windows, where I sometimes want my scripts to run as well.
    - qalmakka 5y ago
      It's also unlikely you have Python installed on Windows though, and there's a non-insignificant change that's an old or random version (probably even Python 2). You can't rely on anything on Windows, so you're definitely going to also install a runtime from Chocolatey or similar.
  - thatwasunusual 5y ago
    > That's why Perl is my go-to choice for glue scripts. It's almost always installed by default My first thought was related to this as well: Perl is _always_ installed, and based on Python's popularity (compared to Perl, at least), I'd assume that Python also was installed by default? That's an honest question, btw: I use Perl quite a lot, Python not so much (at all). :)
jpxw 5y ago
After reading about various vulnerabilities which are the direct result of backwards JS language features - most recently prototype pollution - I’d be hesitant to use JS for writing scripts. Maybe I’m just being paranoid.
- tgsovlerkhgsel 5y ago
  I think a much bigger concern is the ecosystem. The node ecosystem is built around having third-party dependencies for everything, which makes it impossible to meaningfully vet the code you're running, and once a dependency gets hijacked, all its dependencies are screwed too.
azkae 5y ago
Somewhat related, if you are looking to do scripts in Python you should take a look at `sh`: import sh print(sh.ifconfig("eth0")) https://amoffat.github.io/sh/ https://amoffat.github.io/sh/
- ducktective 5y ago
  Is this the ergonomic `subprocess.Popen` alternative?
  - azkae 5y ago
    Yes this wraps subprocess.Popen in a nice API.
- ahaferburg 5y ago
  https://amoffat.github.io/sh/sections/faq.html https://amoffat.github.io/sh/sections/faq.html > Will Windows be supported? > There are no plans to support Windows.
tyingq 5y ago
I suppose it's easy to poke fun at combining the "best" of shell scripting and JS. But, the examples of using await() with child processes and pipes are pretty nice, and less verbose than the Perl or Python equivalents. It does seem well designed for anything where you're orchestrating parallel runs of commands.
- thatwasunusual 5y ago
  For Perl, there is now Future::AsyncAwait.[0] Mojolicius has already adopted it, so things are moving forward at the speed of the magnetic poles shift. :) [0] https://metacpan.org/pod/Future::AsyncAwait https://metacpan.org/pod/Future::AsyncAwait
  - mst 5y ago
    Plus while Mojolicious' support for running external processes is extremely limited because it's not really a goal of the project, http://p3rl.org/IO::Async::Process http://p3rl.org/IO::Async::Process (by the author of the async/await system) is a gloriously complete interface and you can run Mojo::IOLoop and IO::Async::Loop code in the same process just fine.
tgsovlerkhgsel 5y ago
As annoying as having to write "await" in front of everything is (probably should have a variant of $ that's synchronous), the ability to effortlessly implement parallelism is something that all other common scripting (and many "full") languages seem to lack. Even Go, that prides itself on the good support for parallelism, fails once you actually want go get results back. If this were paired with some way to also display the status/output of multiple commands in parallel, this would be the ultimate toolkit to write scripts. If you look at apt, for example: Does it really have to wait for the last package to be downloaded before installing the first? Is there a good reason, on modern SSD-based computers, to not parallelize the installation of multiple packages that don't depend on each other?
- rurban 5y ago
  using sh is usually preferred: somecmd & someothercmd & wait runs the two commands in parallel and waits for both. no need for await. to run in serial replace & with && and skip the wait
  - tgsovlerkhgsel 5y ago
    Huh, neat, didn't know "wait". Thanks! Still doesn't let you easily get the output of the two commands, but a good start.
    - 10000truths 5y ago
      You can just use files as a temporary buffer: (/my/bg_producer1 > /shm/file1) & (/my/bg_producer2 > /shm/file2) & /my/longrunningfgjob wait (/my/consumer1 < /shm/file1) & (/my/consumer2 < /shm/file2) & wait
  - striking 5y ago
    You lose the ability to know whether they succeeded or failed, though. Like most things given to you for free by the shell in shell scripting, it's only like halfway to a real solution. (If you know how many subshells you started and won't get confused by which is which, you can use `wait -n && wait -n && ...` in bash)
jeswin 5y ago
Shameless plug: If you want the exact opposite of this (that is, to use JS from the shell), try https://bashojs.org https://bashojs.org For example: curl 'https://api.openweathermap.org/data/2.5/weather?q=Bangalore&appid=2a125de83c277f4ce0ace5ed482f22b9' | basho --json -j 'x.weather[0].description + ", feels like: " + Math.round(10 * (x.main.feels_like - 273.15))/10 + "°C"'
deleted 5y ago
[deleted]
another-dave 5y ago
First impression is that I wouldn't go near it until it settles down a bit — it was on v1.7 in May and now on version 3 already. Great that it's under active development, but changes to my scripts are infrequent and slow. I wouldn't want to be touching an old script on a server somewhere saying "How do I do X in ZX again?" and everything I find online is now for v15 while I'm still on v3. Maybe if it had plans around a LTS version I'd take a look at that stage. To be honest though I like doing things in bash. You can get pretty far & it's 100% portable. If something's too complicated for bash it's probably time it's not just a simple CLI script anymore, in my experience.
- SevenSigs 5y ago
  > First impression is that I wouldn't go near it until it settles down a bit — it was on v1.7 in May and now on version 3 already. Would you feel better if they called version three v1.9 instead?
  - FinalBriefing 5y ago
    The major version bumps imply breaking changes. Two major versions within a few months implies I might have to refactor my code every couple of months to stay up to date, or face a buildup of tech debt if I want to take advantage of a new feature in the future. If there are no breaking changes, then I would feel better if they called it v1.9.
  - staticassertion 5y ago
    Given that 3 and 1.9 have a completely different semantic meaning, they might.
    - SevenSigs 5y ago
      that depends on the developer(s).. didn't check in this case
- deleted 5y ago
  [deleted]
otabdeveloper4 5y ago
> await await await await Oh yeah, right, because the first thing that comes to mind when writing bash scripts is "I sure wish there was more 'await' noise in all this code!" Anyways: 'async' in the Python and Javascript sense was a mistake, future generations will think we were insane to adopt it. https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
- Vinnl 5y ago
  I haven't looked into Zx yet, but at least for JavaScript that "what color is your function" article also applied before they added the syntactic sugar. If your function was async via a direct callback then the calling function would have to accept a callback too.
- laurent123456 5y ago
  Those are small examples but for more complex scripts I expect you'd mostly have synchronous code, for example to process strings, validate input, etc. and the "await" will become the exception rather than the norm.
- tusharsadhwani 5y ago
  Shameless plug; might wanna look into zxpy :)
- eyelidlessness 5y ago
  Given Promises and callbacks have always been pervasive in JS—that it’s always been heavily asynchronous—I can’t imagine thinking async/await was a mistake. It’s a drastic ergonomic improvement over both, and it also drastically improves debugging and error stacks. Adding it to a language where IO is typically blocking with other ergonomic concurrency mechanisms, that I can imagine being somewhat controversial. It would be good if async weren’t infectious, but as a sibling comment said that was already the case with Promises and callbacks. It would maybe also be better if concurrency was the default behavior and keeping a reference to a promise was the explicit syntax, but I could see downsides to that. It potentially hides/encourages excessive use of expensive calls. And implicit concurrency is a potential source of many bugs.
lixtra 5y ago
await $`dep deploy --branch=${branch}` … await $`mkdir /tmp/${name}` Code like this[1] looks prone to shell injection [1] https://github.com/google/zx https://github.com/google/zx
- pitaj 5y ago
  Template tag functions in JS can intercept the interpolation values passed in. So it's possible they're automatically escaping to prevent shell injection. They could also be parsing out the first word as the command and not using shell execution at all.
  - lixtra 5y ago
    Indeed, escaping is what they seem to do.
- mst 5y ago
  That example is immediately followed by two sentences of text, of which the second sentence is: > The zx package provides useful wrappers around child_process, escapes arguments and gives sensible defaults. lit-html and at least one SQL generating package I've seen use the template string custom interpolation mechanism to auto-escape things - it's a pretty established pattern at this point in javascript land ... but also, yes, until you learn that, it totally does look prone to injection. I've got in the habit of going and finding the template string interpolation function and double checking, because otherwise I find it hard to convince my brain to believe it isn't prone to injection and it interferes with my skim reading the code ;)
hnarn 5y ago
> Bash is great, but when it comes to writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice um… what?
- FinalBriefing 5y ago
  It is _a_ perfect choice. I know a bit of bash, but I'm always looking up how to do things I know are simple in JS. I would love to be able to write scripts in JS because I'm faster with it, and I find it easier to read.
  - gbear0 5y ago
    in case anyone is looking to do this, it's as easy as doing the following #!/usr/bin/env node console.log("Hello World!");
    - rickstanley 5y ago
      In case anyone wondering how this is valid JS: - https://tc39.es/proposal-hashbang/out.html https://tc39.es/proposal-hashbang/out.html - https://github.com/tc39/proposal-hashbang https://github.com/tc39/proposal-hashbang - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#hashbang_comments https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
- soheil 5y ago
  What is the nature of your outrage exactly? Bash is very difficult as a language to write code in. JS is very familiar to a lot of developers and its syntax is orders of magnitude easier to understand than Bash.
  - runawaybottle 5y ago
    The prejudice against JS developers is that they are seen as low class janitorial code monkeys by ‘real programmers’. It hurts these people even more because the language just succeeds and succeeds despite all the hate.
soheil 5y ago
Is this a typo? console.log(`Files count: ${count}`) missing $
- ptx 5y ago
  No, this is an untagged template literal[1], which just generates a string. The other ones are tagged, which means that the template string and the parameters are (separately) passed to the $ function. [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
haolez 5y ago
Is this bundled as a static binary? If yes, what's the binary size?
bokchoi 5y ago
If you haven't yet, give babashka a try! It's fun writing little scripts in clojure. https://github.com/babashka/babashka https://github.com/babashka/babashka
kcartlidge 5y ago
> ZX 3.0 I was disappointed this link didn't go to another ZX Spectrum emulator :(
shell0x 5y ago
This looks horrible to me. The mixing of bash commands in Javascript looks absolutely terrible and running node itself for scripting is a total mess(node_modules, versions). I usually go with bash + set -euxo pipefail and if it becomes too complicated I switch to Python. GoLang/Rust are the perfect tool when distributing tools as a single binare. Where in the world does JS fit in?

189 points by medv 5y ago

I thought this was going to be somehow related to the ZX Spectrum :(

Zx is good if JavaScript is only language you know. If you have ability to program or learn other programming languages, Python with packages like Shell and Plumbum might be better choices for shell scripting tasks that outgrow shell. - Python code with Shell is easier to read ana write - No nees to carry async/await keywords thru the code - Powerful plain text manipulation in the stdlib https://pypi.org/project/python-shell/ https://pypi.org/project/python-shell/ https://plumbum.readthedocs.io/en/latest/ https://plumbum.readthedocs.io/en/latest/

gizdan 5y ago
Wow, plumbum, despite it's unfortunate name, looks awesome. I will definitely try it next time I write a script.
- philsnow 5y ago
  how is the name unfortunate? from the site: > Plumbum (Latin for lead, which was used to create pipes back in the day)
  - marcusramberg 5y ago
    And yet it sounds like a thicc ass.
    - mst 5y ago
      I confess to being quite amazed that you saw an innuendo somewhere I didn't.
  - dogma1138 5y ago
    Yes, but the name choice was quite intentional and probably giggle inducing. This is about as toilet humor as it can get.
    - FroshKiller 5y ago
      According to the Classical Latin pronunciation I learned in school, this is "ploom-boom" rather than "plum-bum."
    - bovermyer 5y ago
      I wonder if your comment says more about how you choose to see the name rather than any inherent meaning. Personally, I only see it as the old name for lead. Interesting to see how others are wired.
    - mst 5y ago
      I guess if you've never learned about the word 'plumbing' before?
  - gizdan 5y ago
    That's fair. My Latin is non-existent, and I clearly was too eager to look at example source instead of reading the description. I'll take my criticism and walk away.
dogma1138 5y ago
Powershell is also available for Linux these days, and well Powershell is awesome. Not sure how I feel about needing Node.JS to run shell scripts…
alexghr 5y ago
Maybe, but the main advantage of `zx` (at least for me) is that I don't have to manage another language environment. I can keep everything contained within Node :)
- dogma1138 5y ago
  The problem is Node, especially all the packages that you’ll end up using under the hood, it’s seems quite risky especially if you don’t have super tight control over your Node SBOM. Would be a bit more interesting to me if it was a JS shell built from the ground up without the entire mess of the Node package dependency baggage.
  - alexghr 5y ago
    While I agree that Node's ecosystem is not the best, it's what we have in the frontend world. If you're working on a Node-based service or React app, you already have to deal with Node and its package ecosystem. Adding a couple of extra MB (or even KB, depending on package reuse) on top of your existing chain of dependencies is a tiny con to the huge benefit of not having to do logic in Bash :)
    - dogma1138 5y ago
      The problem is that you don’t always know what is actually being executed, shell scripting is risky as it is without the rabbit hole that is the Node dependencies system.
  - ptx 5y ago
    There's always WSH if you're on Windows and will settle for ECMAScript 3. No external dependencies or runtime needed.
  - arve0 5y ago
    zx has a small dependency tree, 12 packages marked as "@types" and 36 regular packages[0]. If you are fluent in Node, it should be easy to avoid further dependencies. 0: https://arve0.github.io/npm-download-size/#zx https://arve0.github.io/npm-download-size/#zx
SahAssar 5y ago
> Python code with Shell is easier to read and write That is pretty subjective
- miohtama 5y ago
  Yes. I started programming in JavaScript in 1999 and Python in 2003, so it is my subjective opinion based on mastering the both languages. There are some factual and objevtive features making Python syntax more friendly, but it is a worth of a blog post to explore. And in the end there is no point, as JavaScript maximalists would not care.
IshKebab 5y ago
Maybe but Python is sloooow and its type annotation situation is pretty poor. I would recommend using Deno. It has the following advantages: * Uses Typescript which gives you a great static typing system. * Runs via V8 so it's about a million times faster than Python. * Single statically linked binary with no project setup files required (package.json/dependencies.txt) makes it about a million times easier to deploy than either Node or Python. Especially on Windows. * You can still use third party libraries even without a project file and you get IDE integration. It's the clear winner at this point. Someone has even made a version of zx for it: https://deno.land/x/zx_deno@1.2.2 https://deno.land/x/zx_deno@1.2.2
- tcbasche 5y ago
  I’ve found Python is just fine performance-wise for CLI tools. What bad experiences have you had with it? I’ve mostly used click.
  - watermelon0 5y ago
    Python interpreter by itself is fast, it spawns in a few milliseconds, but it can slow to a crawl if you need libraries, and easily take a few seconds. However, situation is similar with nodejs.
  - IshKebab 5y ago
    Startup time is very slow. I just tested on my system and cold startup was 1 second (!) and hot 180ms. Vs 80ms and 64ms for Node. Also it obviously depends what your CLI tool is doing. Just because something is a CLI tool doesn't mean it doesn't do much stuff and therefore performance doesn't matter. For example Scons is a CLI tool. It's a build system written in Python. Everybody abandoned it because it was dog slow. Mercurial is a CLI tool. It's a VCS written in Python. Most people have abandoned it partly because it is slow (and partly because Github exists). You can read about Mercurial's troubles with Python performance here: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan > Performance is a significant pain point with Python. There are multiple facets to the performance problem: > * Startup overhead > * General performance overhead compared to native code > * GIL interfering with parallel execution > It takes several dozen milliseconds to start a Python interpreter and load the Mercurial Python modules. If you have many extensions loaded, it could take well over 100ms just to effectively get to a Mercurial command's main function. Reports of over 250ms are known. While the command itself may complete in mere milliseconds, Python overhead has already made hg seem non-instantaneous to end-users.
wdroz 5y ago
There is also xonsh[0] if you want an similar experience to zx to mix bash and python. [0] -- https://github.com/xonsh/xonsh https://github.com/xonsh/xonsh

maga 5y ago

And here is the same thing in Deno: https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a1d72 https://gist.github.com/zandaqo/93004fb265146a95aadb28ec851a...

SahAssar 5y ago
That doesn't quote/escape arguments and with zx you can do the shebang so you don't need the import but implementing that in deno shouldn't be too hard I think.
searchableguy 5y ago
https://deno.land/x/dzx@0.2.4 https://deno.land/x/dzx@0.2.4 Here is a more full fleded version.

TeMPOraL 5y ago

What does "DX" stand for in "tons of DX improvements"?

DaGardner 5y ago
Developer Experience I suppose
- kleiba 5y ago
  Oh man, I'm getting too old for this shit...
  - marton78 5y ago
    It's like UX but for developers...
    - TeMPOraL 5y ago
      Which makes no sense if the developers are the users... But in the spirit of "there are only two business models - bundling and unbundling", I guess there are only two marketing tricks: use an old term for a new thing, and introduce a new term for an old thing...
      - zeroxfe 5y ago
        Or more charitably, while the UX/DX distinction isn't very meaningful for this project, there are lots of products (e.g., payments) that target both (non-developer) end users and developers. It helps to be able to separate the personas.
        TeMPOraL 5y ago
        I don't think this holds. As the rule goes, there are only three cardinal numbers: zero, one, many. What are the chances you product has only two meaningful personas - "users" and "developers"? Take that payment system. People who buy things are obviously users, people who write CMS plugins for that system are obviously developers. But what about, say, analysts studying reports from that system? Accountants making sure the money flows where it should? Sysadmins keeping the backend components running? These are all users too. Where UX makes sense as a broad concept, giving a separate acronym to one small subset of potential users... doesn't make sense.

ilaksh 5y ago

I feel like the await keyword everywhere is a bit verbose. One might use something like the gpp preprocessor (hopefully the macros could be defined in a separate file): #!/usr/bin/env zx #define _ await $ #define __ await Promise.all #define _1 process.argv[1] #define _2 process.argv[1] #define _3 process.argv[1] _`cat package.json | grep name` let branch = _`git branch --show-current` _`dep deploy --branch=${branch}` __([ $`sleep 1; echo 1`, $`sleep 2; echo 2`, $`sleep 3; echo 3`, ]) let name = 'foo bar' _`mkdir /tmp/${_1}`

williamdclt 5y ago
I don't think it is very relevant to critique the Javascript syntax on a post about a specific library
- masklinn 5y ago
  This is a node package, so using async/await would have been a specific design decision of the library.
- ilaksh 5y ago
  I am not talking about JavaScript in general. I have been doing mainly JavaScript programming for the last 8 or 9 years or so (before that other languages like C# and C++ for many years). And I use those keywords all the time. For this specific use case, they are trying to replace shell scripts. So I thought maybe some people might be willing to consider abbreviations for the async and await keywords for this case.
deleted 5y ago
[deleted]

mikevm 5y ago

Ugh, typing `await` for every command is terrible. JavaScript's async-by-default model is clearly not a good fit for this use case.

lucideer 5y ago
Not sure what this comment means. Javascript has no "async-by-default" model. The author of this library has chosen to make the "$" function they provide async; this isn't a JS design feature or limitation, it's just the lib author's choice.
lhnz 5y ago
I wonder if they could use some kind of magic to create synchronous versions of all async APIs (e.g. https://giuseppegurgone.com/synchronizing-async-functions/ https://giuseppegurgone.com/synchronizing-async-functions/) Although this would be fighting the language...
tgsovlerkhgsel 5y ago
Requiring it for every command by default sucks, but on the other hand, the ability to easily make scripts async is a huge benefit. I've seen so many cases of processes being synchronous simply because doing it differently would require an unrealistic amount of custom parallelism code.
Too 5y ago
I think the idea is that the majority of the script will be written in pure JS, with only occasional calls out to subprocesses, so “every command” is far from every line. You frequently see the same mental mismatch when shell-scripters try to use Python, and complain how hard it is to pipe with POpen, instead of using the built in string processing features.

satyanash 5y ago

I personally find Ruby's built-in shell features more ergonomic: `cat package.json | grep name` branch = `git branch --show-current` `dep deploy --branch=#{branch}` if $?.success? [ "sleep 1; echo 1", "sleep 2; echo 2", "sleep 3; echo 3", ].map { |cmd| Thread.new { %x(cmd) } }.each(&:join) name = 'foo bar' `mkdir /tmp/#{name}` if $?.success?

eurekin 5y ago
Does this carry over to jruby as well?
chubot 5y ago
How do you do shell escaping of #{name} if it comes from user input?
- pornel 5y ago
  The unnecessary "from user input" qualification is why spaces are kryptonite for most shell-based tools. It doesn't matter where input comes from. Input needs to be escaped for the context it's used in, period. Non-exploitable doesn't mean it's correct.
jitl 5y ago
What I don’t like about backticks in Ruby is that they “ignore” errors in commands you run. It’s up to the program author to remember to check $? for the last executed command’s exit status. And guess how many times the average Ruby script using this feature implements error handling? Usually it’s totally forgotten. To be safe, abstractions that make it easy to shell out must also: - escape all variable interpolations by default using “shellwords” or similar. - throw exceptions on error by default. Backticks in Ruby are very easy to use, but aren’t safe.
- eatonphil 5y ago
  Ruby noob, but surely there's a way to metaprogram and change this to default to throw on non-zero exit?

underdeserver 5y ago

Very cool!

kjgkjhfkjf 5y ago

I'm having a hard time believing that Google uses this much internally. Is github.com/google just a place where Googlers can put personal projects to get additional exposure?

tym0 5y ago
The copyright of the thing Googlers build on their oswn time are usually owned by Google and so it ends up on the Google org if they want it on GitHub. They usually have a disclaimer like "this is not a official Google product..."
- codewiz 5y ago
  This isn't correct: if a Googler writes some code from scratch, which is unrelated to their work, on their personal time, and using their personal laptop... then it's definitely not owned by Google. There could be gray areas where code ownership could be disputed, but in general your employer owns only the intellectual property it paid for. Source: I'm a Googler (until next month).
  - wokwokwok 5y ago
    Are you sure? Not in the “haha, yes of course”, in the “I’ve spoken to HR and have it writing” kind of way. I’ve heard this is not true from others, it is simply true in certain geographical regions where local laws override the contract you signed.
    - mrgriscom 5y ago
      "The provisions of this Agreement requiring disclosure and assignment of Inventions to Google do not apply to any invention that I have developed entirely on my own time without using any of Google Property (as defined below) or Google Confidential Information, except for those inventions that either (i) relate at the time of conception or reduction to practice of the invention to Google’s business, or actual or demonstrably anticipated research or development of Google or (ii) result from any work or services that I performed for Google."

cfeliped 5y ago

In my first job I gave a try at using Python for scripts. It was a terrible idea, as I now had 3 problems instead of 1: 1 - Writing scripts, plus: 2 - Installing Python in every host that needed to run those scripts 3 - Maintaining Python and the necessary dependencies up to date in each host / container. (2) and (3) are trivial when done just once in your own computer, but end up being a huge time sink for larger heterogeneous environments with (possibly) network and security barriers. Moreover, if you are using containers, installing Python will make images unecessarily large. Granted, I'm not a JS person, but I imagine one will have the same issues. Even though bash has its shortcomings, I really appreciate how low maintenance it is as long as you keep scripts small and sane.

qalmakka 5y ago
That's why Perl is my go-to choice for glue scripts. It's almost always installed by default, it has stayed consistent for decades without any breaking changes, its regex support is by far the best around, and its almost as good as the shell at gluing commands together.
- nyanpasu64 5y ago
  I'm less likely to have Perl installed than Python on Windows, where I sometimes want my scripts to run as well.
  - qalmakka 5y ago
    It's also unlikely you have Python installed on Windows though, and there's a non-insignificant change that's an old or random version (probably even Python 2). You can't rely on anything on Windows, so you're definitely going to also install a runtime from Chocolatey or similar.
- thatwasunusual 5y ago
  > That's why Perl is my go-to choice for glue scripts. It's almost always installed by default My first thought was related to this as well: Perl is _always_ installed, and based on Python's popularity (compared to Perl, at least), I'd assume that Python also was installed by default? That's an honest question, btw: I use Perl quite a lot, Python not so much (at all). :)

jpxw 5y ago

After reading about various vulnerabilities which are the direct result of backwards JS language features - most recently prototype pollution - I’d be hesitant to use JS for writing scripts. Maybe I’m just being paranoid.

tgsovlerkhgsel 5y ago
I think a much bigger concern is the ecosystem. The node ecosystem is built around having third-party dependencies for everything, which makes it impossible to meaningfully vet the code you're running, and once a dependency gets hijacked, all its dependencies are screwed too.

azkae 5y ago

Somewhat related, if you are looking to do scripts in Python you should take a look at `sh`: import sh print(sh.ifconfig("eth0")) https://amoffat.github.io/sh/ https://amoffat.github.io/sh/

ducktective 5y ago
Is this the ergonomic `subprocess.Popen` alternative?
- azkae 5y ago
  Yes this wraps subprocess.Popen in a nice API.
ahaferburg 5y ago
https://amoffat.github.io/sh/sections/faq.html https://amoffat.github.io/sh/sections/faq.html > Will Windows be supported? > There are no plans to support Windows.

tyingq 5y ago

I suppose it's easy to poke fun at combining the "best" of shell scripting and JS. But, the examples of using await() with child processes and pipes are pretty nice, and less verbose than the Perl or Python equivalents. It does seem well designed for anything where you're orchestrating parallel runs of commands.

thatwasunusual 5y ago
For Perl, there is now Future::AsyncAwait.[0] Mojolicius has already adopted it, so things are moving forward at the speed of the magnetic poles shift. :) [0] https://metacpan.org/pod/Future::AsyncAwait https://metacpan.org/pod/Future::AsyncAwait
- mst 5y ago
  Plus while Mojolicious' support for running external processes is extremely limited because it's not really a goal of the project, http://p3rl.org/IO::Async::Process http://p3rl.org/IO::Async::Process (by the author of the async/await system) is a gloriously complete interface and you can run Mojo::IOLoop and IO::Async::Loop code in the same process just fine.

tgsovlerkhgsel 5y ago

As annoying as having to write "await" in front of everything is (probably should have a variant of $ that's synchronous), the ability to effortlessly implement parallelism is something that all other common scripting (and many "full") languages seem to lack. Even Go, that prides itself on the good support for parallelism, fails once you actually want go get results back. If this were paired with some way to also display the status/output of multiple commands in parallel, this would be the ultimate toolkit to write scripts. If you look at apt, for example: Does it really have to wait for the last package to be downloaded before installing the first? Is there a good reason, on modern SSD-based computers, to not parallelize the installation of multiple packages that don't depend on each other?

rurban 5y ago
using sh is usually preferred: somecmd & someothercmd & wait runs the two commands in parallel and waits for both. no need for await. to run in serial replace & with && and skip the wait
- tgsovlerkhgsel 5y ago
  Huh, neat, didn't know "wait". Thanks! Still doesn't let you easily get the output of the two commands, but a good start.
  - 10000truths 5y ago
    You can just use files as a temporary buffer: (/my/bg_producer1 > /shm/file1) & (/my/bg_producer2 > /shm/file2) & /my/longrunningfgjob wait (/my/consumer1 < /shm/file1) & (/my/consumer2 < /shm/file2) & wait
- striking 5y ago
  You lose the ability to know whether they succeeded or failed, though. Like most things given to you for free by the shell in shell scripting, it's only like halfway to a real solution. (If you know how many subshells you started and won't get confused by which is which, you can use `wait -n && wait -n && ...` in bash)

jeswin 5y ago

Shameless plug: If you want the exact opposite of this (that is, to use JS from the shell), try https://bashojs.org https://bashojs.org For example: curl 'https://api.openweathermap.org/data/2.5/weather?q=Bangalore&appid=2a125de83c277f4ce0ace5ed482f22b9' | basho --json -j 'x.weather[0].description + ", feels like: " + Math.round(10 * (x.main.feels_like - 273.15))/10 + "°C"'

deleted 5y ago

[deleted]

another-dave 5y ago

First impression is that I wouldn't go near it until it settles down a bit — it was on v1.7 in May and now on version 3 already. Great that it's under active development, but changes to my scripts are infrequent and slow. I wouldn't want to be touching an old script on a server somewhere saying "How do I do X in ZX again?" and everything I find online is now for v15 while I'm still on v3. Maybe if it had plans around a LTS version I'd take a look at that stage. To be honest though I like doing things in bash. You can get pretty far & it's 100% portable. If something's too complicated for bash it's probably time it's not just a simple CLI script anymore, in my experience.

SevenSigs 5y ago
> First impression is that I wouldn't go near it until it settles down a bit — it was on v1.7 in May and now on version 3 already. Would you feel better if they called version three v1.9 instead?
- FinalBriefing 5y ago
  The major version bumps imply breaking changes. Two major versions within a few months implies I might have to refactor my code every couple of months to stay up to date, or face a buildup of tech debt if I want to take advantage of a new feature in the future. If there are no breaking changes, then I would feel better if they called it v1.9.
- staticassertion 5y ago
  Given that 3 and 1.9 have a completely different semantic meaning, they might.
  - SevenSigs 5y ago
    that depends on the developer(s).. didn't check in this case
deleted 5y ago
[deleted]

otabdeveloper4 5y ago

> await await await await Oh yeah, right, because the first thing that comes to mind when writing bash scripts is "I sure wish there was more 'await' noise in all this code!" Anyways: 'async' in the Python and Javascript sense was a mistake, future generations will think we were insane to adopt it. https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

Vinnl 5y ago
I haven't looked into Zx yet, but at least for JavaScript that "what color is your function" article also applied before they added the syntactic sugar. If your function was async via a direct callback then the calling function would have to accept a callback too.
laurent123456 5y ago
Those are small examples but for more complex scripts I expect you'd mostly have synchronous code, for example to process strings, validate input, etc. and the "await" will become the exception rather than the norm.
tusharsadhwani 5y ago
Shameless plug; might wanna look into zxpy :)
eyelidlessness 5y ago
Given Promises and callbacks have always been pervasive in JS—that it’s always been heavily asynchronous—I can’t imagine thinking async/await was a mistake. It’s a drastic ergonomic improvement over both, and it also drastically improves debugging and error stacks. Adding it to a language where IO is typically blocking with other ergonomic concurrency mechanisms, that I can imagine being somewhat controversial. It would be good if async weren’t infectious, but as a sibling comment said that was already the case with Promises and callbacks. It would maybe also be better if concurrency was the default behavior and keeping a reference to a promise was the explicit syntax, but I could see downsides to that. It potentially hides/encourages excessive use of expensive calls. And implicit concurrency is a potential source of many bugs.

lixtra 5y ago

await $`dep deploy --branch=${branch}` … await $`mkdir /tmp/${name}` Code like this[1] looks prone to shell injection [1] https://github.com/google/zx https://github.com/google/zx

pitaj 5y ago
Template tag functions in JS can intercept the interpolation values passed in. So it's possible they're automatically escaping to prevent shell injection. They could also be parsing out the first word as the command and not using shell execution at all.
- lixtra 5y ago
  Indeed, escaping is what they seem to do.
mst 5y ago
That example is immediately followed by two sentences of text, of which the second sentence is: > The zx package provides useful wrappers around child_process, escapes arguments and gives sensible defaults. lit-html and at least one SQL generating package I've seen use the template string custom interpolation mechanism to auto-escape things - it's a pretty established pattern at this point in javascript land ... but also, yes, until you learn that, it totally does look prone to injection. I've got in the habit of going and finding the template string interpolation function and double checking, because otherwise I find it hard to convince my brain to believe it isn't prone to injection and it interferes with my skim reading the code ;)

hnarn 5y ago

> Bash is great, but when it comes to writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice um… what?

FinalBriefing 5y ago
It is _a_ perfect choice. I know a bit of bash, but I'm always looking up how to do things I know are simple in JS. I would love to be able to write scripts in JS because I'm faster with it, and I find it easier to read.
- gbear0 5y ago
  in case anyone is looking to do this, it's as easy as doing the following #!/usr/bin/env node console.log("Hello World!");
  - rickstanley 5y ago
    In case anyone wondering how this is valid JS: - https://tc39.es/proposal-hashbang/out.html https://tc39.es/proposal-hashbang/out.html - https://github.com/tc39/proposal-hashbang https://github.com/tc39/proposal-hashbang - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#hashbang_comments https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
soheil 5y ago
What is the nature of your outrage exactly? Bash is very difficult as a language to write code in. JS is very familiar to a lot of developers and its syntax is orders of magnitude easier to understand than Bash.
- runawaybottle 5y ago
  The prejudice against JS developers is that they are seen as low class janitorial code monkeys by ‘real programmers’. It hurts these people even more because the language just succeeds and succeeds despite all the hate.

soheil 5y ago

Is this a typo? console.log(`Files count: ${count}`) missing $

ptx 5y ago
No, this is an untagged template literal[1], which just generates a string. The other ones are tagged, which means that the template string and the parameters are (separately) passed to the $ function. [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

haolez 5y ago

Is this bundled as a static binary? If yes, what's the binary size?

bokchoi 5y ago

If you haven't yet, give babashka a try! It's fun writing little scripts in clojure. https://github.com/babashka/babashka https://github.com/babashka/babashka

kcartlidge 5y ago

> ZX 3.0 I was disappointed this link didn't go to another ZX Spectrum emulator :(

shell0x 5y ago

This looks horrible to me. The mixing of bash commands in Javascript looks absolutely terrible and running node itself for scripting is a total mess(node_modules, versions). I usually go with bash + set -euxo pipefail and if it becomes too complicated I switch to Python. GoLang/Rust are the perfect tool when distributing tools as a single binare. Where in the world does JS fit in?