3 ms·
> But where bash scripts fall down is the constant re-parsing of text, data has to be passed through the kernel, and extra processes (when you use xargs, which
by kbenson 9y ago
> But where bash scripts fall down is the constant re-parsing of text, data has to be passed through the kernel, and extra processes (when you use xargs, which tends to happen for complicated tasks.) This adds up super quick.
You do't always have to use xargs. There are many ways to pipeline shell scripts. For one, you can process the input stream and add markers to make it easier to identify items later in the stream, or convert it to a well defined format for a similar outcome.
Additionally, any extra resource and time usage from xargs parsing arguments (which I think is likely very small), can probably be easily recouped by the fact that you can use a single xargs flag to run commands in parallel. I've effectively used xargs to automate connecting to through SSH and running commands on hundreds of servers when automating processes.
Finally, you're setting up a false dichotomy between shell pipelines and programming. I work in Perl for my day job, and write complex, well structured and designed Perl in that job. I also write shell scripts and Perl one liners. More importantly, I often use the Perl I've written in my one liners by loading the module and calling the functions I've defined there. In this way I get the best of both worlds, where I get to use pipelines to quickly munge text, and then pass it to more complex well written routines to process. This can often be parallelized as well.