I chuckle whenever is see the acronym RDD, I can't shake the joke a while back that it stands for 'resume-driven development.' With the largest EC2 instances h

I chuckle whenever is see the acronym RDD, I can't shake the joke a while back that it stands for 'resume-driven development.' With the largest EC2 instances having 1TB of RAM, many problems that used to require distributed compute can be more efficiently run on one machine with a variety of software: R, Python/Pandas, scikit, julia, etc. When you've cleaned and filtered your data to meet the problem space, you may well find you don't have big data -- and be relieved that you dont need to hire distributed systems engineers to keep everything running. There are important use cases however, for example small-world graphs.