Where did you get the $10M figure for GPT-3? That sounds awfully cheap considering the cost of compute alone: one estimate was $4.6M for a single training run [

Where did you get the $10M figure for GPT-3? That sounds awfully cheap considering the cost of compute alone: one estimate was $4.6M for a single training run [0], while other sources [1] put it at $12M per run. I highly doubt that OpenAI nailed the training process right on the second or even first go respectively (according to your figure). So even conservative estimates put the compute cost alone at least one order of magnitude higher than your figure of $10M. [0] https://lambdalabs.com/blog/demystifying-gpt-3/#:~:text=But%20to%20put%20things%20into,for%20a%20single%20training%20run https://lambdalabs.com/blog/demystifying-gpt-3/#:~:text=But%.... [1] https://venturebeat.com/2020/06/01/ai-machine-learning-openai-gpt-3-size-isnt-everything/ https://venturebeat.com/2020/06/01/ai-machine-learning-opena...