4 ms·
As someone at a large corp who just had a 3 day battle to get a new VM with 2 gig of ram and a shared CPU I suddenly feel like crying. I guess I have no real u
by boyter 15y ago
As someone at a large corp who just had a 3 day battle to get a new VM with 2 gig of ram and a shared CPU I suddenly feel like crying.
I guess I have no real use for something of this magnitude, but I would still like to get my hands on a seriously large cluster just for the sake of expanding my experience.
I don't suppose anyone knows of a cost effective way of doing so? I have spun up my own Hadoop clusters before but with only 4 machines its not exactly exciting.
- jonburs 15y agoAmazon -- you can use Elastic Map Reduce [1] (the easiest way to run a transient Hadoop cluster) or spin up a bunch of EC2 nodes and deploy your own stack. Both methods can provide you access to very powerful hardware [2]. (Disclaimer: I work for Amazon, but not in AWS, and these are my own personal opinions.) [1] http://aws.amazon.com/elasticmapreduce/ http://aws.amazon.com/elasticmapreduce/ [2] http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/
- epistasis 15y agoI don't work for Amazon, have hundreds of non-AWS CPUs at my disposal, and still prefer AWS, as long as you don't have to do much I/O. Once there's a few terabytes of data or more, it's best to be off of AWS, because you'll never be able to saturate your CPUs. Even for smaller datasets of 10s of gigabytes, it's kind of iffy.