To generate 100 Terabytes of data in parallel ... on Hadoop https://github.com/hortonworks/hive-testbench/blob/hive14/tpcds-setup.sh#L116 https://github.com/ho

To generate 100 Terabytes of data in parallel ... on Hadoop https://github.com/hortonworks/hive-testbench/blob/hive14/tpcds-setup.sh#L116 https://github.com/hortonworks/hive-testbench/blob/hive14/tp... The shell script generates a Makefile and the Makefile runs the hadoop commands, so that the parallel dep handling is entirely handed off to Make. This make it super easy to run 2 parallel workloads at all times - unlike xargs -P 2, this is much more friendly towards complex before/after deps and failure handling.