5 ms·

For me, the most surprising one was paste. paste allowed me to interleave to streams or to split out a single stream into two columns. I'd been writing custom

For me, the most surprising one was paste. paste allowed me to interleave to streams or to split out a single stream into two columns. I'd been writing custom scripting monstrosities before I discovered paste: $ paste <( echo -e 'foo\nbar' ) <( echo -e 'baz\nqux' ) foo baz bar qux $ echo -e 'foo\nbar\nbaz\nqux' | paste - - foo bar baz qux I wonder what other unix gems I've been missing...

bloopernova 7y ago
Paste is quite wonderful. It adds a lot of flexibility in output and should definitely be more widely known.
loeg 7y ago
The related join(1) and comm(1) are oft-missed and occasionally helpful.
- cptnapalm 7y ago
  I'm going through the AWK book and join is in there. I was unhappily surprised to find that using a tab as a delimiter is painful with join. Variations to overcome this which I've bumped into: join -t $'\t' file1 file2 (BASH only, I think.) join -t '<CTRL+v><Tab>' file1 file2 join -t "`echo '\t'`" file1 file2 Why 'join -t '\t' file1 file2' is apparently beyond the pale has me mystified.
  - loeg 7y ago
    Given that the delimiter must be a single character and the intended use is in shells, I am equally mystified as to why control codes must be passed literally. Join is smart enough to reject 2 character delimiters, but could easily grok escapes if it so chose: $ join -t '\t' join: illegal tab character specification
  - JdeBP 7y ago
    See https://unix.stackexchange.com/a/65819/5132 https://unix.stackexchange.com/a/65819/5132 for some thinking on making every individual program have an escape sequence parser, and why that syntax is not specific nor original to the Bourne Again shell.
    - cptnapalm 7y ago
      Found a link in your link to something by a Sven Mascheck and found out there is a printf command. Despite having started using Linux in 1997, I still find new things.
JdeBP 7y ago
I would guess, from experience of people doing things the hard way, at: * John A. Kunze's jot and rs * John Kerl's mlr ("Miller") * jq * join and comm, as mentioned * fmt * ex And, given what you just wrote: * printf
- AdieuToLogic 7y ago
  A handy one to add to your list is: seq[0] EDIT: I didn't see that you had already included 'jot' so removed it from my original reply. 0 - https://www.freebsd.org/cgi/man.cgi?query=seq&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html https://www.freebsd.org/cgi/man.cgi?query=seq&apropos=0&sekt...
  - JdeBP 7y ago
    I didn't include it because in my experience people don't overlook seq, whereas they do overlook jot. I only included things where I've encountered people overlooking such tools and doing things the hard way.
- pge 7y ago
  Do I remember correctly that “jot” was also the name of a gui text editor on unix systems as well (different from the program referred to in the parent comment)? I think I used it in the early 90s on an SGI workstation running sun solaris OS.
  - mixmastamyk 7y ago
    Yes, I remember a jot being the “notepad” of SGI workstations. Didn’t take me long to move on to nedit instead.
  - loeg 7y ago
    Are you thinking of joe? https://joe-editor.sourceforge.io/ https://joe-editor.sourceforge.io/
jerf 7y ago
Recently I found myself wanting to stop a program after a certain timeout. I found myself thinking "sure there's a UNIX program to do this in shell?" There is. And it's called... brace yourself... "timeout". 20+ years in UNIX, never encountered it. Part of GNU coreutils.
- saagarjha 7y ago
  It’s unfortunately not part of POSIX, which means that BusyBox can’t get the options right or even stay consistently wrong :(
- mekster 7y ago
  Sometimes good to use for cron jobs so that the previous one won't keep running on top of the current run. You should debug for it to not happen though but probably better than seeing your system choke with dozens of instances even slowing the jobs more. There's also "run-one" that allows you to control which job should survive instead of just killing the old one. http://manpages.ubuntu.com/manpages/trusty/man1/run-one.1.html http://manpages.ubuntu.com/manpages/trusty/man1/run-one.1.ht...
  - majewsky 7y ago
    If you're using systemd timers, you get this behavior for free because those timers only activate service units. If the service is already active, nothing happens. (Of course, this can be surprising in a different way when you expect a new job to start, but it does not because a previous one is still lingering. Not saying it's better or worse, just pointing out for those who don't know.)
thedance 7y ago
paste is an important part of the shell wizard's diet. paste with one input from `yes` is quite useful.

bloopernova 7y ago

Paste is quite wonderful. It adds a lot of flexibility in output and should definitely be more widely known.

loeg 7y ago

The related join(1) and comm(1) are oft-missed and occasionally helpful.

cptnapalm 7y ago
I'm going through the AWK book and join is in there. I was unhappily surprised to find that using a tab as a delimiter is painful with join. Variations to overcome this which I've bumped into: join -t $'\t' file1 file2 (BASH only, I think.) join -t '<CTRL+v><Tab>' file1 file2 join -t "`echo '\t'`" file1 file2 Why 'join -t '\t' file1 file2' is apparently beyond the pale has me mystified.
- loeg 7y ago
  Given that the delimiter must be a single character and the intended use is in shells, I am equally mystified as to why control codes must be passed literally. Join is smart enough to reject 2 character delimiters, but could easily grok escapes if it so chose: $ join -t '\t' join: illegal tab character specification
- JdeBP 7y ago
  See https://unix.stackexchange.com/a/65819/5132 https://unix.stackexchange.com/a/65819/5132 for some thinking on making every individual program have an escape sequence parser, and why that syntax is not specific nor original to the Bourne Again shell.
  - cptnapalm 7y ago
    Found a link in your link to something by a Sven Mascheck and found out there is a printf command. Despite having started using Linux in 1997, I still find new things.

JdeBP 7y ago

I would guess, from experience of people doing things the hard way, at: * John A. Kunze's jot and rs * John Kerl's mlr ("Miller") * jq * join and comm, as mentioned * fmt * ex And, given what you just wrote: * printf

AdieuToLogic 7y ago
A handy one to add to your list is: seq[0] EDIT: I didn't see that you had already included 'jot' so removed it from my original reply. 0 - https://www.freebsd.org/cgi/man.cgi?query=seq&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html https://www.freebsd.org/cgi/man.cgi?query=seq&apropos=0&sekt...
- JdeBP 7y ago
  I didn't include it because in my experience people don't overlook seq, whereas they do overlook jot. I only included things where I've encountered people overlooking such tools and doing things the hard way.
pge 7y ago
Do I remember correctly that “jot” was also the name of a gui text editor on unix systems as well (different from the program referred to in the parent comment)? I think I used it in the early 90s on an SGI workstation running sun solaris OS.
- mixmastamyk 7y ago
  Yes, I remember a jot being the “notepad” of SGI workstations. Didn’t take me long to move on to nedit instead.
- loeg 7y ago
  Are you thinking of joe? https://joe-editor.sourceforge.io/ https://joe-editor.sourceforge.io/

jerf 7y ago

Recently I found myself wanting to stop a program after a certain timeout. I found myself thinking "sure there's a UNIX program to do this in shell?" There is. And it's called... brace yourself... "timeout". 20+ years in UNIX, never encountered it. Part of GNU coreutils.

saagarjha 7y ago
It’s unfortunately not part of POSIX, which means that BusyBox can’t get the options right or even stay consistently wrong :(
mekster 7y ago
Sometimes good to use for cron jobs so that the previous one won't keep running on top of the current run. You should debug for it to not happen though but probably better than seeing your system choke with dozens of instances even slowing the jobs more. There's also "run-one" that allows you to control which job should survive instead of just killing the old one. http://manpages.ubuntu.com/manpages/trusty/man1/run-one.1.html http://manpages.ubuntu.com/manpages/trusty/man1/run-one.1.ht...
- majewsky 7y ago
  If you're using systemd timers, you get this behavior for free because those timers only activate service units. If the service is already active, nothing happens. (Of course, this can be surprising in a different way when you expect a new job to start, but it does not because a previous one is still lingering. Not saying it's better or worse, just pointing out for those who don't know.)

thedance 7y ago

paste is an important part of the shell wizard's diet. paste with one input from `yes` is quite useful.