You’re right but I’d just like to add, GPT-3 probably required 1000’s of GPUs. OpenAI is known to have the largest cluster 16k+ A100 GPUs and most of them were

You’re right but I’d just like to add, GPT-3 probably required 1000’s of GPUs. OpenAI is known to have the largest cluster 16k+ A100 GPUs and most of them were used for the major model training.