4 ms·
This is correct, in the job configuration only the initial data is loaded from s3, afterwards you can chain mapreduce steps that run off hdfs on the local machi
by pskomoroch 18y ago
This is correct, in the job configuration only the initial data is loaded from s3, afterwards you can chain mapreduce steps that run off hdfs on the local machines.
Step 1:
"-input", "s3n://elasticmapreduce/samples/similarity/lastfm/input/",
"-output", "hdfs:///home/hadoop/output2/",
Step 2:
"-input", "hdfs:///home/hadoop/output2/",
"-output", "hdfs:///home/hadoop/output3/",
etc..
There is no explicit support for mounting data from EBS yet, but in theory I think you can hack something together in the initial mapper to mount an EBS volume to be used in the second map task...