2 ms·

any idea what the cost for training GPT-5 is going to be? Dario Amodei of Anthropic threw a ballpark number[0] for training future LLMs which is frankly astoni

any idea what the cost for training GPT-5 is going to be? Dario Amodei of Anthropic threw a ballpark number[0] for training future LLMs which is frankly astonishing > The models that are in training now and that will come out at various times later this year or early next year are closer in cost to $1 billion. So that’s already happening. And then I think in 2025 and 2026, we’ll get more towards $5 or $10 billion. but maybe there have been some improvement in efficiency? [0] https://www.nytimes.com/2024/04/12/opinion/ezra-klein-podcast-dario-amodei.html?showTranscript=1 https://www.nytimes.com/2024/04/12/opinion/ezra-klein-podcas...

dragonwriter 2y ago
Because more money will deliver more quality even with efficiency gains as long as we don't hit a ceiling on ability to improve with more resources, the cost of new top-end SotA models will scale with hiw much capital is attracted to AI, and efficiency will not reduce top-end SotA model costs. (It may reduce “good enough” model costs for many tasks, but that doesn’t effect the top of the market.)