3 ms·
Down voted :) I think this is my first comment. Anyway, yes, I'm the CTO for a small startup and we regularly spin up 500 node clusters on AWS for data proces
by dwhitney 15y ago
Down voted :) I think this is my first comment. Anyway, yes, I'm the CTO for a small startup and we regularly spin up 500 node clusters on AWS for data processing. Yes that's a quarter of what Facebook is claiming, and yeah I see that their nodes have 32 gigs of RAM and we typically use either the 8 or 16 gig instance types, which are smaller, but machines of the type described in that article are available, and we're 8 person company with 3 developers, which means it's not unlikely that my claim is wrong (i.e. a larger company could easily exceed the capacity we need). Amazon allows this type of thing with ElasticMapReduce. It's normal. Do I need to supply proof? Naw, this is a bulletin board.