3 ms·
I believe significant share of GPT-3 cost is machine-hours that were spent training this model - months of hundreds top-tier NVidia machines. Edit: estimates r
by coolspot 6y ago
I believe significant share of GPT-3 cost is machine-hours that were spent training this model - months of hundreds top-tier NVidia machines.
Edit: estimates range from $2-$5MM to $15MM
https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_the_cost_of_training_gpt3/ https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_t...
- zaptrem 6y agoI thought MS was giving them Azure GPU instances for free?
- coolspot 6y ago“Free”, which is basically Microsoft, as an investor, has paid those $4-$15MM minus ~30% of margin they make over pure operational costs. Electricity was burned and NVidia GPUs that could have been doing paid job were doing GPT-3 training instead.
- jychang 6y ago$15mm is a rounding error for MSFT