3 ms·
"But to put things into perspective, GPT-3 175B model required 3.14E23 FLOPS of computing for training. Even at theoretical 28 TFLOPS for V100 and lowest 3 year
by snypher 3y ago
"But to put things into perspective, GPT-3 175B model required 3.14E23 FLOPS of computing for training. Even at theoretical 28 TFLOPS for V100 and lowest 3 year reserved cloud pricing we could find, this will take 355 GPU-years and cost $4.6M for a single training run."
https://lambdalabs.com/blog/demystifying-gpt-3/ https://lambdalabs.com/blog/demystifying-gpt-3/