3 ms·
Try $100/hour for big LLM's... And you're probably going to need a fleet of 16 machines unless you want to quantize it and do inference only.
by londons_explore 3y ago
Try $100/hour for big LLM's... And you're probably going to need a fleet of 16 machines unless you want to quantize it and do inference only.
- sp332 3y agoWhere can you even find a machine for $100/hour? The most expensive one on this list is just over $5 and is definitely overkill for running a 40B model. https://www.paperspace.com/gpu-cloud-comparison https://www.paperspace.com/gpu-cloud-comparison