4 ms·
Wow. Well, I wish I would have known abut GNU parallel before I wrote that massive bash script a few years ago. Thanks for posting this
by earthscienceman 6y ago
Wow. Well, I wish I would have known abut GNU parallel before I wrote that massive bash script a few years ago. Thanks for posting this.
- abhgh 6y agoIts a very handy utility. Sometimes when I am lazy to write proper multiprocessing support in my codes, I get some work started anyway by (1) quickly providing a basic CLI version of my code (2) throwing in a bunch of code-invocation commands in a file and calling GNU parallel on it.
- yjftsjthsd-h 6y agoLearning more unix tools tends to lead to that feeling a lot. The last few decades of devs building tools generally means that most obvious problems have a solution, but of course one usually only learns about that solution after hacking together their own version:) One tip: Look up moreutils; it's a nice consolidation of things that make a lot of things easier. (One gotcha: moreutils ships its own different parallel tool.)
- betaby 6y agoxargs can do most of the common tasks perl parallel does, see xars -P
- RMPR 6y agoImho, the main advantage of Parallel is that it can execute the tasks on multiple hosts, straight from the man page: > GNU parallel is a shell tool for executing jobs in parallel using one or more computers. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables.
- mechanical_jane 6y agoThis gives an (artificial) example of the power of GNU Parallel over xargs: https://unix.stackexchange.com/questions/405552/using-xargs-instead-of-gnu-parallel https://unix.stackexchange.com/questions/405552/using-xargs-...
- arminiusreturns 6y agoIt's great for scaling text smashing and bashing imho.
- gspr 6y agoIt's an absolute godsend for plowing through data-parallel computations.