3 ms·
> awk+xargs can replace hours of work Including machine hours of work. Wasn't there a famous story of replacing a Hadoop cluster with an awk script (which was
by 5e92cb50239222b 5y ago
> awk+xargs can replace hours of work
Including machine hours of work.
Wasn't there a famous story of replacing a Hadoop cluster with an awk script (which was a couple orders of magnitude faster)?
Oh yes, there was: https://news.ycombinator.com/item?id=17135841 https://news.ycombinator.com/item?id=17135841
- dapids 5y agoIn fairness it's xargs that is providing the command parallelization, not awk, but I agree both combined are a good match.
- genewitch 5y agoIf one considers the idea of map reduce to be taking a set of data and ending up with a subset that is relevant, I've used tons of simple things to do that, and never Hadoop. I think parsing logs to find pain areas or potential exploit/exfil is a map reduce job, for instance, and grep or awk can manage that just fine.