That was my first thought, I do ETL work at a large insurance company and we use shell tools almost exclusively unless we need to do more complex transformation

That was my first thought, I do ETL work at a large insurance company and we use shell tools almost exclusively unless we need to do more complex transformations. Unix tools are really low overhead and work extremely well for this task. Also, you can parallelize a lot of them with xargs: https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...