The data input bottleneck is small. If you're going to do something in n*log(n), you should be fine. If you're going something in n^2, you're fucked anyway. Map

The data input bottleneck is small. If you're going to do something in n*log(n), you should be fine. If you're going something in n^2, you're fucked anyway. Map Reduce is your friend. On that note, I only learned about Hive since coming to facebook. They made and open sourced it. Check it out: http://wiki.apache.org/hadoop/Hive http://wiki.apache.org/hadoop/Hive