4 ms·
How much does it cost to run GPT-3?
Hi everyone. I need some help calculating the costs of running GPT-3 based on what they have released (transformer with 175 billion params).
I'm looking for two metrics:
1- How much VRAM/RAM and CPU power would you need to run the trained model with decent response time (1-5 seconds) for a single input, and what would be the financial costs? (basically, I want to calculate that, even if OpenAI made the model available to the public, how much would they have to spend on hardware/cloud infrastructure costs)
2- How much does a single inference cost in terms of FLOPS and eventually in monetary value? (Basically, if say, I was running a customer service chatbot on GPT-3 with an average response length of 150 characters, how much would each response cost me)
Does anyone have any idea how to calculate this?
- Crossfold 6y agoI don't think GPT 3 is free for use. It's limited to some corporates only.
- bendee983 6y agoI know it's not open to the public. I'm trying to estimate the costs of running/inference based on what we know about the computation costs of transformers and scaling them to 175 billion params