3 ms·
even ignoring distillation, so long as hardware or ml get better over time, training a new model from scratch is cheaper the later you do it
by adgjlsfhk1 7mo ago
even ignoring distillation, so long as hardware or ml get better over time, training a new model from scratch is cheaper the later you do it
- ef3dfd 7mo agoYep the poster is assuming efficiencies will not come. Absolutely they will. And this is a huge problem for OAI - given Google is targeting vertical integration, they will acquire a cost-advantage. As long as the model performance is good enough, they will kick OAI and Anthropic out in the long-run. The valuations of OAI and Anthropic are nonsense. A true valuation would incorporate failure risk, which is natural for startups/fast growing and money losing firms. Anyone who takes them serious is incredibly delusional.
- aurareturn 7mo agoIf hardware gets better over time, they also get better for OpenAI.
- vlovich123 7mo agoIt does, but for what it’s worth they have tech and financial tech of their past to pay down. This is how startups disrupt incumbents