4 ms·
You're off by an order of magnitude and omit the caveats to that cost estimation. From the article: > The cost of training the model from scratch using our cod
by the8472 7y ago
You're off by an order of magnitude and omit the caveats to that cost estimation. From the article:
> The cost of training the model from scratch using our code is about $50k. It’s important to note this figure is the estimated value of the cloud compute, and does not reflect the much smaller intrinsic costs involved
- p1esk 7y agoThey edited the article after I left a comment there. The original text stated they spent $500k to run all the hyperparameter search experiments to replicate OpenAI results. Only after they did all that work you can run their code for $50k.