3 ms·
Found some interesting contrasting statements from 2015 [1]: > The company performed big data jobs early on in the cloud, using Amazon's Elastic Map Reduce ser
by gregmac 9y ago
Found some interesting contrasting statements from 2015 [1]:
> The company performed big data jobs early on in the cloud, using Amazon's Elastic Map Reduce service.
> “That [EMR] got us to a place where we think about what data we collect. At the time it was really inefficient from a cost perspective, but there weren’t any creative constraints,”
> The team eventually decided to bring it in house. “Our Hadoop cluster wields a huge amount of flexibility," said Rembetsy. "We can get a lot done on bare metal, and we know very much what we’re looking for. Cloud services, that’s where efficiencies fall away. Spin up and spin down doesn’t have the same value proposition.”
[1] http://www.datacenterknowledge.com/archives/2015/05/11/how-etsy-optimized-data-center-infrastructure-for-efficiency http://www.datacenterknowledge.com/archives/2015/05/11/how-e...
- interskh 9y agoEMR is really expensive, especially you are using spot instances. It is roughly $2 for the EMR tax on $1 instance