2 ms·
> Big shell pipelines are actually blazingly fast, since each command in the pipeline runs in parallel. Yeah, there was an article a few years back about that,
by kbenson 9y ago
> Big shell pipelines are actually blazingly fast, since each command in the pipeline runs in parallel.
Yeah, there was an article a few years back about that, Command-line tools can be faster than your Hadoop cluster [1], where the punchline is that because of the inherent stream processing because of pipelining, the shell pipeline was 235 times faster than Hadoop.
1: https://news.ycombinator.com/item?id=8908462 https://news.ycombinator.com/item?id=8908462