5 ms·
Map-Reduce With Ruby Using Apache Hadoop
- xal 16y agoNot quite as nice as the incredible mrjob, but it's a good start in a great language.
- mattyb 16y agohttp://engineeringblog.yelp.com/2010/10/mrjob-distributed-computing-for-everybody.html http://engineeringblog.yelp.com/2010/10/mrjob-distributed-co...
- gfodor 16y agoAnother potentially easier way to do this is to use elastic map reduce: http://aws.amazon.com/elasticmapreduce/ http://aws.amazon.com/elasticmapreduce/ This will automatically spin up and tear down the instances your job uses, and will set up the hadoop configuration to access your S3 account instead of HDFS. So, you upload your input to S3, run your job (in this case, a Hadoop streaming job since it's a Ruby script), and the output will be saved to S3 for you to download. The only script you need to write is the actual map reduce job, and you kick off the job by uploading the script to s3 and running the elastic-mapreduce command line tool.