2 ms·
Hm, there are some interesting numbers in there. "Map Reduce Usage at Google: 3.5m jobs/year averaging 488 machines each & taking ~8 min ... Big Table Usage at
by pmorici 17y ago
Hm, there are some interesting numbers in there.
"Map Reduce Usage at Google: 3.5m jobs/year averaging 488 machines each & taking ~8 min ...
Big Table Usage at Google: 500 clusters with largest having 70PB, 30+ GB/s I/O"
So to run 3.5 million jobs at 8 minutes each on 488 machines that means they would need at least 26,069 machines to complete those map reduce jobs in a year.
Similarly if you deduce that for their largest storage cluster they are using their previously described commodity hardware approach and at the moment the sweet spot for drives is at about 1TB with 1 or two drives per machine that is between 36,700 and 73,400 machines in their largest storage cluster. That seems like a lot.