It's probably a better idea to use Hadoop 1.0.4 or even 2.0. Lot's of little annoyances are solved there. Monitoring is also completely missing: We use Icinga/

It's probably a better idea to use Hadoop 1.0.4 or even 2.0. Lot's of little annoyances are solved there. Monitoring is also completely missing: We use Icinga/Nagios and Ganglia. Especially Ganglia is invaluable to adjust the configuration for optimal machine usage in my experience. Another point worth considering is Security. Hadoop by default is secured like NFS. That means any user that is able to create an hdfs or hadoop user on a machine that has access to the NameNode can delete your HDFS. Hadoop can use Kerberos for security. Also consider adding Snappy Compression to your setup, it speeds up the shuffle phase. Last but not least - I've found these slides about Hadoop Tuning invaluable: http://www.slideshare.net/cloudera/mr-perf http://www.slideshare.net/cloudera/mr-perf @meinuelzen: We use Oracle JDK 7 with en_US.UTF-8 and ntpd on all machines. Ubuntu 10.04 / Ubuntu 12.04 but the OS should not matter. Lot's of RAM and lot's of disks are more important.