3 ms·
"Reading the OpenAI GPT-3 paper. Impressive performance on many few-shot language tasks. The cost to train this 175 billion parameter language model appears to
by flaviojuvenal 6y ago
"Reading the OpenAI GPT-3 paper. Impressive performance on many few-shot language tasks. The cost to train this 175 billion parameter language model appears to be staggering: Nearly $12 million dollars in compute based on public cloud GPU/TPU cost models (200x the price of GPT-2)"
https://twitter.com/eturner303/status/1266264358771757057 https://twitter.com/eturner303/status/1266264358771757057
- zaroth 6y agoNearly $12 million dollars in compute based on public cloud GPU/TPU cost models (200x the price of GPT-2)" Oh! So only about $1 million on bare metal.