5 ms·
A fun trick is using the `paste` command with process substitution to combine lines of output from different commands. ./cmd1 outputs: a b c ./cmd
by vimax 5y ago
A fun trick is using the `paste` command with process substitution to combine lines of output from different commands.
./cmd1
outputs:
a
b
c
./cmd2
outputs:
1
2
3
paste <(./cmd1) <(./cmd2)
outputs:
a 1
b 2
c 3
You can then use xargs to use each line as arguments for another command:
paste <(./cmd1) <(./cmd2) | xargs -L1 ./cmd3
calls:
./cmd3 a 1
./cmd3 b 2
./cmd3 c 3
- sillysaurusx 5y agoShould’ve been named zip. Is there an unzip? But of course it can’t be named zip because that’s the compressor, so I’m really asking whether there’s an unpaste. I guess the closest is awk {print $1;}, but it makes me wince every time I write it. (I already know that would give syntax errors as written here.)
- vimax 5y agoDepending on the data, the right options for xargs would work, and there's the pr command I haven't used much.
- sillysaurusx 5y agoHmm. Does xargs give you a way to skip the first arg? xargs -n 2 I {} almost works, but you’d need a way to extract a specific arg out of the {}. Which I just realized I don’t know how to do. “(at 0 {})” would be a nice construction. But then of course now you’re writing xargs -n 2 -I {} echo (at 1 {}) instead of just awk {print $2}, so it’s mission failed successfully.
- quicklime 5y agoIf you're just looking for a single column, then as zamfi mentioned, cut does the job: $ paste <(printf "a\nb\nc\n") <(printf "1\n2\n3\n") | cut -f2 1 2 3 It makes sense that "cut" and "paste" would be inverse operations, but too bad that GUIs repurposed these words for clipboard operations. But awk can do both, and does so in a single pass: $ paste <(printf "a\nb\nc\n") <(printf "1\n2\n3\n") | awk '{ print $1 > "letters"; print $2 > "numbers" }' $ cat letters a b c $ cat numbers 1 2 3
- sillysaurusx 5y agoWhat! I didn’t know awk could do that within single quotes. The heck, it implements a subset of sh? Thanks!
- camhutch 5y agoawk has some basic redirection features. gawk documentation: https://web.mit.edu/gnu/doc/html/gawk_6.html#SEC41 https://web.mit.edu/gnu/doc/html/gawk_6.html#SEC41 posix documentation: https://pubs.opengroup.org/onlinepubs/009695299/utilities/awk.html#tag_04_06_13_10 https://pubs.opengroup.org/onlinepubs/009695299/utilities/aw...
- stewartbutler 5y agoKinda-sorta related, you can pipe to a multivariable loop: paste <(printf "a\nb\nc\n") <(printf "1\n2\n3\n") | while read x y; do echo "$y -- $x"; done Kinda silly for this example, but very useful in things like `kubectl` pipelines -- you can do something like this: kubectl get pods --all-namespaces | awk '{print $1 " " $2}' | grep "doesntexist" | while read x y; do kubectl delete pod $y --namespace $x --wait=false; done This lets you list all pods in a cluster, grep a selection of them by name, and delete them in a batch, regardless of which namespace they are in, while shuffling the order of args into different flags.
- sillysaurusx 5y agoMost useful comment I’ve read in months, for me. Cheers!
- forty 5y agoSorry, but my shellcheck OCD forces me to say "read without -r will mangle backslashes." ;)
- CornCobs 5y agoTIL you can pipe into files in awk. I've never come across this feature somehow. Is it commonly used?
- zamfi 5y agoif you know the separator, you can use `cut` -- not an unzip per se (doesn't give you the full separated lists, without calling it multiple times anyway) but approximately equal to the awk line
- teraflop 5y agoNow you've got me curious which came first, "paste" or "zip". Python's zip function was added in PEP 201 [1], which credits the Haskell 98 report with the name. The same function appears in all prior versions of the Haskell language that I could find, dating back to Haskell 1.0 in 1990. On the other hand, the GNU coreutils version of paste[2] has copyright dates going all the way back to 1984, which presumably means it was derived from UNIX System V or an even earlier version. [1]: https://www.python.org/dev/peps/pep-0201/ https://www.python.org/dev/peps/pep-0201/ [2]: https://github.com/coreutils/coreutils/blob/master/src/paste.c https://github.com/coreutils/coreutils/blob/master/src/paste...
- nicoburns 5y agoIt's possible a lisp of some sort predated the Haskell version
- LukeShu 5y agoThe FreeBSD man page for `paste` claims it first appeared in Unix 32V (1979; derived from v7 Unix, and ancestral to both 4BSD and SysV).
- mprovost 5y agoHere's the commit from 1978: https://github.com/dspinellis/unix-history-repo/blob/Bell-32V-Snapshot-Development/usr/src/cmd/paste.c https://github.com/dspinellis/unix-history-repo/blob/Bell-32... The comments refer to an "old 127 paste command" but I have no idea what that is.
- LukeShu 5y agoI hadn't pursued it further than seeing that it was added in an 1978 commit. Looking at it further, there are a few answers at https://minnie.tuhs.org/pipermail/tuhs/2020-January/019955.html https://minnie.tuhs.org/pipermail/tuhs/2020-January/019955.h... > GWRL stands for Gottfried W. R. Luderer, the author of cut(1) and paste(1), probably around 1978. Those came either from PWB or USG I can verify that neither cut nor past were in PWB 1: https://minnie.tuhs.org/cgi-bin/utree.pl?file=PWB1 https://minnie.tuhs.org/cgi-bin/utree.pl?file=PWB1 > Also "127" was the internal department number for the Computer Science Research group at Bell Labs where UNIX originated
- fragmede 5y agoalias awk1="awk {print $1;}" Counting to 9, inclusive, is left as an exercise for the reader.
- boring_twenties 5y agoawp () { local a a=() while [[ -z ${1:#-*} ]] do a=($a $1) shift done local n n=$1 shift awk $a '{print $'$n'}' $* }
- sillysaurusx 5y agoAmusingly, I apparently added these to scrap https://github.com/shawwn/scrap https://github.com/shawwn/scrap many years ago. I named them 1st, 2nd, 3rd, and 4th. So I guess I never needed to count to 5, but I'm still two ahead of Valve. https://www.youtube.com/watch?v=jpw2ebhTSKs https://www.youtube.com/watch?v=jpw2ebhTSKs EDIT: Actually, I'd never committed 4th. It was just sitting here in my local scrap. So I guess I counted to three, realized at some later date I needed four, copied 3rd to 4th, then never committed. Oh. Apparently I also have 'nth', which was exactly what I wanted: > echo a b c d | nth 0 3 a d The solution I came up with at the time: #!/bin/bash c="" for n in "$@" do n=$((n+1)) args="${args}${c}\$$n" c=', ' done awk "{ print ${args}; }" Nowadays I'd write it like: awk "{ print $(echo "$@" | args math '1+{}' | replace '^' '$' | trim | joinlines ', '); }" But I admit both are equally cryptic, and the former might even be more readable.
- cerved 5y ago> Is there an unzip? xargs printf
- throwawayboise 5y agoHow do you know that the outputs a, b, and c correspond to outputs 1, 2, and 3 and whether they will always occur in the same orders. This technique seems like it's full of possibly invalid assumptions.