3 ms·
Reminds me of https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-fas
by 0zymandiass 5y ago
Reminds me of
https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...
Cluster computing can be useful, but until you're talking about petabytes of data, it probably isn't helping you
- TylerE 5y agoKind of reminds me of the early days of like, Hadoop, when it was all the rage. Then people realized they could do most that stuff in a python script on a single machine in less time because of all the bookkeeping overhead and complexity.
- meritt 5y agoToday they still do it in python except it’s now running on Spark across sprawling expensive clusters to do complex tasks like transform text or coalesce two fields.