but he wrote a web scraping program for mining the data, bonded it together using wget, launched several aws instances with hadoop, and deployed a mapreduce job

but he wrote a web scraping program for mining the data, bonded it together using wget, launched several aws instances with hadoop, and deployed a mapreduce job for cleaning up the data. It took three weeks and $200 usd, but damn, modern technology is awesome, it would be impossible to do this just 10 years ago. Thanks google, thanks amazon.