4 ms·
Sure, you also just need 3600 machines.
by neilc 17y ago
Sure, you also just need 3600 machines.
- diego 17y agoYou don't have to sort a terabyte in 62 seconds, you could do it on 10 machines and it would probably take you less than a day. If you are a company that needs to sort a petabyte once in a while and your data resides on the Amazon cloud, you could get 3600 high-cpu EC2 instances and do it for about $12k. A petabyte is on the order of 100k bytes for every person on the planet. If you have that much data that you need to sort in less than a day, you can afford it.
- neilc 17y agoI'd actually be curious to see what would happen if you tried to sort a petabyte with 3800 EC2 nodes. I wonder how much worse your performance would be? EC2 instance-local storage is pretty slow by default (the "first-write" problem), but if you used multiple EBS volumes on each node you might be able to get pretty good I/O performance.
- sanswork 17y agoJust a small correction. Using their mapreduce large instances it would cost you less that $3500.