5 ms·
> My friends at Replicate told me that a simple rule of thumb for A100 cloud costs is $1/hour. AWS charges $32/hr for an 8xA100s (p4d.24xlarge) which comes out
by agnokapathetic 4y ago
> My friends at Replicate told me that a simple rule of thumb for A100 cloud costs is $1/hour.
AWS charges $32/hr for an 8xA100s (p4d.24xlarge) which comes out to $4/hour/gpu. Yes you can get lower pricing with a 3 year reservation but thats not what this question is asking.
You also need 256 nodes to be colocated on the same fabric -- which AWS will do for you but only if you reserve for years.
- sebzim4500 4y agoMaybe they are using spot instances? $1/hr is about right for those.
- thewataccount 4y agoAWS certainly isn't the cheapest for this, did they mention using AWS? Lamdba Labs is 12$/hr for 8xA100's, and there's others relatively close to this price on demand, I assume you can get a better deal if you contact them for a large project. Replicate themselves rent out GPU time so I assume they would definitely know as that's almost certainly the core of their business.
- IanCal 4y agoLambda labs charges about 11-12/hr for 8xA100.
- celestialcheese 4y agolambdalabs will let you do on-demand 8xa100 @ 80GB VRAM/GPU for $12/hr, or reserved @ $10.86/hr 8xA100 @ 40gb for $8/hr Replicate friend isn't far off.
- pavelstoev 4y agomodel-depending, you can train on lesser (cheaper) GPUs but system-level optimizations are needed. Which is what we provide at centml.ai