4 ms·
Can independent researchers not afford $6? That’s what 512 GPUs for 1.5 minutes costs on EC2, using spot P2.16xl with 16GPUs each. The price would be $12 if yo
by FakeComments 8y ago
Can independent researchers not afford $6?
That’s what 512 GPUs for 1.5 minutes costs on EC2, using spot P2.16xl with 16GPUs each. The price would be $12 if you had to use on-demand instances.
It does start to add up — if you wanted to do 2 runs per hour, 8 hours a day, 250 days per year (ie, a “full work year”), it would cost $48k. (And you’d have done 4000 such experiments.)
All of this is to say, I think you might be exaggerating this as the “death of independent researchers”.
A single card can still do meaningful inference on a model trained this way, and can do useful prototyping before you deploy your fanout for a few minutes and $20.
- nightski 8y agoYou probably won't get a 56 Gbps network on AWS as used in the article. So it's probably going to take quite a bit longer than 2 minutes. This also ignores all data transfer & storage costs.
- FakeComments 8y agoIt’s 25Gbps between instances, and NVLINK between GPUs on the same instance. It’s really a pretty comparable setup. There’s no network cost within a region, and you’re probably not downloading terabytes of data out of AWS — so you’re talking about a few dollars a month, as opposed to tens a day in GPU cost. (You’d also have 23TB of RAM in your cluster, if that’s enough for you to use as storage.)
- jph00 8y agoAlso AWS has p3dn.24xlarge with 100Gbps :)