3 ms·
Pricing depends a lot on how much memory your job requires[1] and how much processing each record requires -- i.e. the pricing is more sensitive to usage. As a
by bcongdon 8y ago
Pricing depends a lot on how much memory your job requires[1] and how much processing each record requires -- i.e. the pricing is more sensitive to usage.
As a very rough estimate, for a light-to-medium load of 1Tb, the cost would probably be in the ballpark of ~$0.50. AWS's own reference MR framework[2] (which is mostly a tech demo) quotes prices in a similar order of magnitude.
Corral isn't great for processing-heavy MR jobs, as Lambda pricing rises quickly if you need a lot of memory or take a lot of time with each record. But, for small-ish low-overhead jobs, it can pretty easily beat the pricing and hassle of using something like EMR.
[1]: https://aws.amazon.com/lambda/pricing/#Lambda_pricing_details https://aws.amazon.com/lambda/pricing/#Lambda_pricing_detail...
[2]: https://github.com/awslabs/lambda-refarch-mapreduce/ https://github.com/awslabs/lambda-refarch-mapreduce/
- shaunray 8y agoHi OP, I am the person that made the most recent changes to the AWS Labs refarch. I had been working on a golang version and wanted to clean up the python one. Sunil the original author used the AMPLABS benchmark to calculate the results table. I was planning on updating it with the 1 and 5 node test. Would be happy to include Corral as well.