It is interesting to compare this with Apache Spark, which won the open source Graysort competition of 2014 [1]. Apache Spark completed a 1PB sort on 190 EC2 (

It is interesting to compare this with Apache Spark, which won the open source Graysort competition of 2014 [1]. Apache Spark completed a 1PB sort on 190 EC2 (i2.8xlarge) instances in 234 mins. Google did their 2011 1PB sort on 8000 computers in 33 minutes. Moore's law is at work here and the results aren't very comparable but it would be interesting to see a head-to-head test on similarly speced clusters. [1] https://databricks.com/blog/2014/10/10/spark-petabyte-sort.html https://databricks.com/blog/2014/10/10/spark-petabyte-sort.h...