3 ms·
With regards parallel jobs, what's wrong with sticking a "&" at the end of the line? Everything he listed is pretty trivial. It comes down to the idea that th
by RiderOfGiraffes 17y ago
With regards parallel jobs, what's wrong with sticking a "&" at the end of the line? Everything he listed is pretty trivial.
It comes down to the idea that the command line is a "non-discoverable" interface. If you don't know what you want, there's pretty much no way to find it.
Any suggestions how to fix that? Google solves a lot of them, but not all. Having said that, I've had fair success recently with just typing entire questions into it as if it were a programmer I'm talking to ...
- kyleburton 17y agoI can run a set # of jobs at a time, working through the list until the set is exhausted - a simple pool of 'workers'. It buffers child process output and emits it when the child completes - this keeps concurrent process output from being as confusing. Those were features I wanted at the time and '&' with wait didn't provide them.
- RiderOfGiraffes 17y agoAh, simply saying "parallelize" put me on the wrong track. I didn't realize you wanted to limit the number running simultaneously. Cool you discovered that xargs can do it - I didn't know that. I have, however, done this before. I have a "load_wait" command which waits for machine load to drop below a certain level before launching the command. A simple "sleep 1" between instances of that then lets me launch thousands of commands without killing the machine. sleep 1 ; load_wait 3 <command1> & sleep 1 ; load_wait 3 <command2> & sleep 1 ; load_wait 3 <command3> & sleep 1 ; load_wait 3 <command4> & ... I then launch those via "system" from inside awk and I'm done. The commands tend to be in muscle memory.
- tptacek 17y ago& either runs the job in the background once (and relies on your to manually schedule), or runs every job simultaneously and kills the box. What you need is a queue. Hence, the tool.
- dreish 17y agoI learned probably around 80% of the shell commands I use through apropos. For example, if I'd been searching for the "pipe thingy" lbrandy's blog entry starts out searching for, I might have typed "apropos pipe", and lo and behold, there it is at the bottom of the list: ... pipe(2) - create descriptor pair for interprocess communication pipe(8) - Postfix delivery to external command rotatelogs(8) - Piped logging program to rotate Apache logs tee(1) - pipe fitting "pipe fitting" isn't the most useful description, but given how few other things there are, it's not too many man pages to read, especially after culling out the things I already knew about back when I didn't yet know about tee(1).
- ars 17y agoOn my machine (debian lenny) it's listed as: tee (2) - duplicating pipe content tee (1) does not show up in the search for pipe, but it's description is: tee (1) - read from standard input and write to standard output and files
- jimbokun 17y ago"Google solves a lot of them, but not all." Interesting that you would say that, as I have a friend who works for Google who would like to use machine learning techniques to make a better command line. It also seems to me that Google's "one box" interface for Chrome, where pretty much no matter what you want to do you just start typing something in the address bar, is maybe the best approximation for what you would want a really good command line interface to look like.